2023-07-14T11:25:03.757 INFO:root:teuthology version: 0.0.1.dev145+gaa6a80e 2023-07-14T11:25:03.759 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337437 --name rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi --verbose --owner scheduled_rishabh@teuthology --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{no-subvolume} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/iozone}} -- /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337437/orig.config.yaml 2023-07-14T11:25:03.814 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T11:25:03.916 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337437 branch: wip-rishabh-2023Jul13 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{no-subvolume} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/iozone}} email: null first_in_suite: false job_id: '7337437' kernel: client: branch: testing flavor: default kdb: true sha1: a440c79eee0258ae07e4988d8690fe6187b8b7a3 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-rishabh-2023Jul13 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: 3fc0089cc1082313a98853d48b892acdc1385b8b ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: 3fc0089cc1082313a98853d48b892acdc1385b8b kclient: mntopts: - ms_mode=secure - wsync syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-2023Jul13 sha1: 3fc0089cc1082313a98853d48b892acdc1385b8b owner: scheduled_rishabh@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 sha1: 3fc0089cc1082313a98853d48b892acdc1385b8b sleep_before_teardown: 0 suite: fs suite_branch: wip-rishabh-2023Jul13 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 3fc0089cc1082313a98853d48b892acdc1385b8b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi017.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= smithi093.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= smithi097.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1M - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6M3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - suites/iozone.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: aa6a80e6cc655fff48452a7eca02070e61d074fc timestamp: 2023-07-14_10:26:42 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.23459 2023-07-14T11:25:03.916 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa; will attempt to use it 2023-07-14T11:25:03.917 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa/tasks 2023-07-14T11:25:03.917 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-07-14T11:25:03.919 INFO:teuthology.task.internal:Checking packages... 2023-07-14T11:25:03.948 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '3fc0089cc1082313a98853d48b892acdc1385b8b' 2023-07-14T11:25:03.949 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-07-14T11:25:03.949 INFO:teuthology.packaging:ref: None 2023-07-14T11:25:03.950 INFO:teuthology.packaging:tag: None 2023-07-14T11:25:03.950 INFO:teuthology.packaging:branch: wip-rishabh-2023Jul13 2023-07-14T11:25:03.950 INFO:teuthology.packaging:sha1: 3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-14T11:25:03.950 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-rishabh-2023Jul13 2023-07-14T11:25:04.457 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-4757.g3fc0089c 2023-07-14T11:25:04.457 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-07-14T11:25:04.464 INFO:teuthology.task.internal:no buildpackages task found 2023-07-14T11:25:04.464 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-07-14T11:25:04.470 INFO:teuthology.task.internal:Saving configuration 2023-07-14T11:25:04.490 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-07-14T11:25:04.496 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-14T11:25:04.526 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi017.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337437', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 11:20:05.074994', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-14T11:25:04.549 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi093.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337437', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 11:20:05.073765', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-14T11:25:04.571 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi097.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337437', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 11:20:05.072104', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-14T11:25:04.572 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-07-14T11:25:04.578 INFO:teuthology.task.internal:roles: ubuntu@smithi017.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-07-14T11:25:04.579 INFO:teuthology.task.internal:roles: ubuntu@smithi093.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-07-14T11:25:04.579 INFO:teuthology.task.internal:roles: ubuntu@smithi097.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-07-14T11:25:04.579 INFO:teuthology.run_tasks:Running task console_log... 2023-07-14T11:25:04.701 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fac4f472550>, signals=[15]) 2023-07-14T11:25:04.702 INFO:teuthology.run_tasks:Running task internal.connect... 2023-07-14T11:25:04.711 INFO:teuthology.task.internal:Opening connections... 2023-07-14T11:25:04.711 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi017.front.sepia.ceph.com 2023-07-14T11:25:04.714 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T11:25:04.785 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi093.front.sepia.ceph.com 2023-07-14T11:25:04.786 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T11:25:04.853 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi097.front.sepia.ceph.com 2023-07-14T11:25:04.854 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T11:25:04.920 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-07-14T11:25:04.927 DEBUG:teuthology.orchestra.run.smithi017:> uname -m 2023-07-14T11:25:04.948 INFO:teuthology.orchestra.run.smithi017.stdout:x86_64 2023-07-14T11:25:04.949 DEBUG:teuthology.orchestra.run.smithi017:> cat /etc/os-release 2023-07-14T11:25:05.009 INFO:teuthology.orchestra.run.smithi017.stdout:NAME="Red Hat Enterprise Linux" 2023-07-14T11:25:05.010 INFO:teuthology.orchestra.run.smithi017.stdout:VERSION="8.6 (Ootpa)" 2023-07-14T11:25:05.010 INFO:teuthology.orchestra.run.smithi017.stdout:ID="rhel" 2023-07-14T11:25:05.010 INFO:teuthology.orchestra.run.smithi017.stdout:ID_LIKE="fedora" 2023-07-14T11:25:05.011 INFO:teuthology.orchestra.run.smithi017.stdout:VERSION_ID="8.6" 2023-07-14T11:25:05.011 INFO:teuthology.orchestra.run.smithi017.stdout:PLATFORM_ID="platform:el8" 2023-07-14T11:25:05.011 INFO:teuthology.orchestra.run.smithi017.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-14T11:25:05.012 INFO:teuthology.orchestra.run.smithi017.stdout:ANSI_COLOR="0;31" 2023-07-14T11:25:05.012 INFO:teuthology.orchestra.run.smithi017.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-14T11:25:05.012 INFO:teuthology.orchestra.run.smithi017.stdout:HOME_URL="https://www.redhat.com/" 2023-07-14T11:25:05.013 INFO:teuthology.orchestra.run.smithi017.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-14T11:25:05.013 INFO:teuthology.orchestra.run.smithi017.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-14T11:25:05.013 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:25:05.013 INFO:teuthology.orchestra.run.smithi017.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-14T11:25:05.014 INFO:teuthology.orchestra.run.smithi017.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-14T11:25:05.014 INFO:teuthology.orchestra.run.smithi017.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-14T11:25:05.014 INFO:teuthology.orchestra.run.smithi017.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-14T11:25:05.015 INFO:teuthology.lock.ops:Updating smithi017.front.sepia.ceph.com on lock server 2023-07-14T11:25:05.043 DEBUG:teuthology.orchestra.run.smithi093:> uname -m 2023-07-14T11:25:05.064 INFO:teuthology.orchestra.run.smithi093.stdout:x86_64 2023-07-14T11:25:05.065 DEBUG:teuthology.orchestra.run.smithi093:> cat /etc/os-release 2023-07-14T11:25:05.124 INFO:teuthology.orchestra.run.smithi093.stdout:NAME="Red Hat Enterprise Linux" 2023-07-14T11:25:05.125 INFO:teuthology.orchestra.run.smithi093.stdout:VERSION="8.6 (Ootpa)" 2023-07-14T11:25:05.125 INFO:teuthology.orchestra.run.smithi093.stdout:ID="rhel" 2023-07-14T11:25:05.126 INFO:teuthology.orchestra.run.smithi093.stdout:ID_LIKE="fedora" 2023-07-14T11:25:05.126 INFO:teuthology.orchestra.run.smithi093.stdout:VERSION_ID="8.6" 2023-07-14T11:25:05.126 INFO:teuthology.orchestra.run.smithi093.stdout:PLATFORM_ID="platform:el8" 2023-07-14T11:25:05.127 INFO:teuthology.orchestra.run.smithi093.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-14T11:25:05.127 INFO:teuthology.orchestra.run.smithi093.stdout:ANSI_COLOR="0;31" 2023-07-14T11:25:05.127 INFO:teuthology.orchestra.run.smithi093.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-14T11:25:05.128 INFO:teuthology.orchestra.run.smithi093.stdout:HOME_URL="https://www.redhat.com/" 2023-07-14T11:25:05.128 INFO:teuthology.orchestra.run.smithi093.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-14T11:25:05.128 INFO:teuthology.orchestra.run.smithi093.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-14T11:25:05.131 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-14T11:25:05.131 INFO:teuthology.orchestra.run.smithi093.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-14T11:25:05.131 INFO:teuthology.orchestra.run.smithi093.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-14T11:25:05.131 INFO:teuthology.orchestra.run.smithi093.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-14T11:25:05.131 INFO:teuthology.orchestra.run.smithi093.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-14T11:25:05.131 INFO:teuthology.lock.ops:Updating smithi093.front.sepia.ceph.com on lock server 2023-07-14T11:25:05.160 DEBUG:teuthology.orchestra.run.smithi097:> uname -m 2023-07-14T11:25:05.180 INFO:teuthology.orchestra.run.smithi097.stdout:x86_64 2023-07-14T11:25:05.181 DEBUG:teuthology.orchestra.run.smithi097:> cat /etc/os-release 2023-07-14T11:25:05.239 INFO:teuthology.orchestra.run.smithi097.stdout:NAME="Red Hat Enterprise Linux" 2023-07-14T11:25:05.239 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION="8.6 (Ootpa)" 2023-07-14T11:25:05.239 INFO:teuthology.orchestra.run.smithi097.stdout:ID="rhel" 2023-07-14T11:25:05.239 INFO:teuthology.orchestra.run.smithi097.stdout:ID_LIKE="fedora" 2023-07-14T11:25:05.239 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION_ID="8.6" 2023-07-14T11:25:05.240 INFO:teuthology.orchestra.run.smithi097.stdout:PLATFORM_ID="platform:el8" 2023-07-14T11:25:05.240 INFO:teuthology.orchestra.run.smithi097.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-14T11:25:05.240 INFO:teuthology.orchestra.run.smithi097.stdout:ANSI_COLOR="0;31" 2023-07-14T11:25:05.240 INFO:teuthology.orchestra.run.smithi097.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-14T11:25:05.240 INFO:teuthology.orchestra.run.smithi097.stdout:HOME_URL="https://www.redhat.com/" 2023-07-14T11:25:05.240 INFO:teuthology.orchestra.run.smithi097.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-14T11:25:05.240 INFO:teuthology.orchestra.run.smithi097.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-14T11:25:05.241 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-14T11:25:05.241 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-14T11:25:05.241 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-14T11:25:05.241 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-14T11:25:05.241 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-14T11:25:05.241 INFO:teuthology.lock.ops:Updating smithi097.front.sepia.ceph.com on lock server 2023-07-14T11:25:05.260 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-07-14T11:25:05.272 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-07-14T11:25:05.278 INFO:teuthology.task.internal:Checking for old test directory... 2023-07-14T11:25:05.279 DEBUG:teuthology.orchestra.run.smithi017:> test '!' -e /home/ubuntu/cephtest 2023-07-14T11:25:05.282 DEBUG:teuthology.orchestra.run.smithi093:> test '!' -e /home/ubuntu/cephtest 2023-07-14T11:25:05.284 DEBUG:teuthology.orchestra.run.smithi097:> test '!' -e /home/ubuntu/cephtest 2023-07-14T11:25:05.301 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-07-14T11:25:05.308 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-07-14T11:25:05.309 DEBUG:teuthology.orchestra.run.smithi017:> test -z $(ls -A /var/lib/ceph) 2023-07-14T11:25:05.341 DEBUG:teuthology.orchestra.run.smithi093:> test -z $(ls -A /var/lib/ceph) 2023-07-14T11:25:05.343 DEBUG:teuthology.orchestra.run.smithi097:> test -z $(ls -A /var/lib/ceph) 2023-07-14T11:25:05.371 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-07-14T11:25:05.451 INFO:teuthology.run_tasks:Running task kernel... 2023-07-14T11:25:05.474 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': 'a440c79eee0258ae07e4988d8690fe6187b8b7a3'} 2023-07-14T11:25:05.475 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2023-07-14T11:25:05.475 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2023-07-14T11:25:05.476 DEBUG:teuthology.orchestra.run.smithi017:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-14T11:25:05.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T11:25:05.497 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-07-14T11:25:06.071 DEBUG:teuthology.task.kernel:sha1 for client.0 is a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-14T11:25:06.072 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "a440c79eee0258ae07e4988d8690fe6187b8b7a3"... 2023-07-14T11:25:06.072 DEBUG:teuthology.orchestra.run.smithi017:> uname -r 2023-07-14T11:25:06.092 INFO:teuthology.orchestra.run.smithi017.stdout:4.18.0-372.9.1.el8.x86_64 2023-07-14T11:25:06.093 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-14T11:25:06.093 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-07-14T11:25:06.094 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum upgrade -y linux-firmware 2023-07-14T11:25:07.354 INFO:teuthology.orchestra.run.smithi017.stdout:Updating Subscription Management repositories. 2023-07-14T11:25:07.354 INFO:teuthology.orchestra.run.smithi017.stdout:Unable to read consumer identity 2023-07-14T11:25:09.025 INFO:teuthology.orchestra.run.smithi017.stdout:Copr repo for python3-asyncssh owned by ceph 9.0 kB/s | 3.5 kB 00:00 2023-07-14T11:25:16.167 INFO:teuthology.orchestra.run.smithi017.stdout:Extra Packages for Enterprise Linux 2.2 MB/s | 16 MB 00:07 2023-07-14T11:25:19.850 INFO:teuthology.orchestra.run.smithi017.stdout:lab-extras 405 kB/s | 24 kB 00:00 2023-07-14T11:25:20.635 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2023-07-14T11:25:20.636 INFO:teuthology.orchestra.run.smithi017.stdout:Nothing to do. 2023-07-14T11:25:20.637 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2023-07-14T11:25:20.743 INFO:teuthology.task.kernel:Downloading kernel a440c79eee0258ae07e4988d8690fe6187b8b7a3 on client.0... 2023-07-14T11:25:20.744 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-14T11:25:21.179 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/a440c79eee0258ae07e4988d8690fe6187b8b7a3/centos/8/flavors/default/x86_64/ 2023-07-14T11:25:21.179 DEBUG:teuthology.orchestra.run.smithi017:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc1_ga440c79eee02-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/a440c79eee0258ae07e4988d8690fe6187b8b7a3/centos/8/flavors/default/x86_64/ --input-file=- 2023-07-14T11:25:21.182 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi017.front.sepia.ceph.com to complete... 2023-07-14T11:29:27.667 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14 11:29:27 URL:https://2.chacra.ceph.com/r/kernel/testing/a440c79eee0258ae07e4988d8690fe6187b8b7a3/centos/8/flavors/default/x86_64/kernel-6.5.0_rc1_ga440c79eee02-1.x86_64.rpm [380684764/380684764] -> "/tmp/kernel.x86_64.rpm" [1] 2023-07-14T11:29:27.667 INFO:teuthology.orchestra.run.smithi017.stderr:FINISHED --2023-07-14 11:29:27-- 2023-07-14T11:29:27.668 INFO:teuthology.orchestra.run.smithi017.stderr:Total wall clock time: 4m 6s 2023-07-14T11:29:27.668 INFO:teuthology.orchestra.run.smithi017.stderr:Downloaded: 1 files, 363M in 4m 6s (1.48 MB/s) 2023-07-14T11:29:27.670 INFO:teuthology.task.kernel:Installing kernel a440c79eee0258ae07e4988d8690fe6187b8b7a3 on client.0... 2023-07-14T11:29:27.671 DEBUG:teuthology.orchestra.run.smithi017:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-07-14T11:29:29.339 INFO:teuthology.orchestra.run.smithi017.stdout:Verifying... ######################################## 2023-07-14T11:29:29.397 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing... ######################################## 2023-07-14T11:29:29.487 INFO:teuthology.orchestra.run.smithi017.stdout:Updating / installing... 2023-07-14T11:30:10.128 INFO:teuthology.orchestra.run.smithi017.stdout:kernel-6.5.0_rc1_ga440c79eee02-1 ######################################## 2023-07-14T11:30:35.086 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi017.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-07-14T11:30:35.086 DEBUG:teuthology.orchestra.run.smithi017:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-07-14T11:30:35.144 INFO:teuthology.orchestra.run.smithi017.stdout:/boot/System.map-6.5.0-rc1-ga440c79eee02 2023-07-14T11:30:35.144 INFO:teuthology.orchestra.run.smithi017.stdout:/boot/config-6.5.0-rc1-ga440c79eee02 2023-07-14T11:30:35.144 INFO:teuthology.orchestra.run.smithi017.stdout:/boot/vmlinuz-6.5.0-rc1-ga440c79eee02 2023-07-14T11:30:35.145 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02 2023-07-14T11:30:35.145 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel 2023-07-14T11:30:35.145 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch 2023-07-14T11:30:35.145 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86 2023-07-14T11:30:35.145 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto 2023-07-14T11:30:35.145 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-07-14T11:30:35.145 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-07-14T11:30:35.146 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-07-14T11:30:35.146 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-07-14T11:30:35.146 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-07-14T11:30:35.146 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-07-14T11:30:35.146 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-07-14T11:30:35.147 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/crc32c-intel.ko 2023-07-14T11:30:35.147 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-07-14T11:30:35.147 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-07-14T11:30:35.147 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-07-14T11:30:35.147 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-07-14T11:30:35.147 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/serpent-avx2.ko 2023-07-14T11:30:35.148 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-07-14T11:30:35.148 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-07-14T11:30:35.148 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-07-14T11:30:35.148 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-07-14T11:30:35.148 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-07-14T11:30:35.148 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-07-14T11:30:35.148 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-07-14T11:30:35.149 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel 2023-07-14T11:30:35.149 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/cpu 2023-07-14T11:30:35.149 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/cpu/mce 2023-07-14T11:30:35.149 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-07-14T11:30:35.149 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm 2023-07-14T11:30:35.149 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm/kvm-amd.ko 2023-07-14T11:30:35.150 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm/kvm-intel.ko 2023-07-14T11:30:35.150 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm/kvm.ko 2023-07-14T11:30:35.150 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto 2023-07-14T11:30:35.150 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/ansi_cprng.ko 2023-07-14T11:30:35.150 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/anubis.ko 2023-07-14T11:30:35.150 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/arc4.ko 2023-07-14T11:30:35.150 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx 2023-07-14T11:30:35.150 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_memcpy.ko 2023-07-14T11:30:35.151 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_pq.ko 2023-07-14T11:30:35.151 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_raid6_recov.ko 2023-07-14T11:30:35.151 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_tx.ko 2023-07-14T11:30:35.151 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_xor.ko 2023-07-14T11:30:35.151 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/raid6test.ko 2023-07-14T11:30:35.151 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/authenc.ko 2023-07-14T11:30:35.151 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/authencesn.ko 2023-07-14T11:30:35.152 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/blake2b_generic.ko 2023-07-14T11:30:35.152 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/blowfish_common.ko 2023-07-14T11:30:35.152 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/blowfish_generic.ko 2023-07-14T11:30:35.152 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/camellia_generic.ko 2023-07-14T11:30:35.152 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cast5_generic.ko 2023-07-14T11:30:35.152 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cast6_generic.ko 2023-07-14T11:30:35.152 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cast_common.ko 2023-07-14T11:30:35.153 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/ccm.ko 2023-07-14T11:30:35.153 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cmac.ko 2023-07-14T11:30:35.153 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/crc32_generic.ko 2023-07-14T11:30:35.153 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/crypto_engine.ko 2023-07-14T11:30:35.153 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/crypto_user.ko 2023-07-14T11:30:35.153 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cts.ko 2023-07-14T11:30:35.154 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/des_generic.ko 2023-07-14T11:30:35.154 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/dh_generic.ko 2023-07-14T11:30:35.154 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/echainiv.ko 2023-07-14T11:30:35.154 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/essiv.ko 2023-07-14T11:30:35.154 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/fcrypt.ko 2023-07-14T11:30:35.154 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/khazad.ko 2023-07-14T11:30:35.154 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/lz4.ko 2023-07-14T11:30:35.154 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/lz4hc.ko 2023-07-14T11:30:35.155 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/md4.ko 2023-07-14T11:30:35.155 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/michael_mic.ko 2023-07-14T11:30:35.155 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/pcbc.ko 2023-07-14T11:30:35.155 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/pcrypt.ko 2023-07-14T11:30:35.156 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/rmd160.ko 2023-07-14T11:30:35.156 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/seed.ko 2023-07-14T11:30:35.156 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/serpent_generic.ko 2023-07-14T11:30:35.156 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/tcrypt.ko 2023-07-14T11:30:35.157 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/tea.ko 2023-07-14T11:30:35.157 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/twofish_common.ko 2023-07-14T11:30:35.157 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/twofish_generic.ko 2023-07-14T11:30:35.158 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/vmac.ko 2023-07-14T11:30:35.158 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/wp512.ko 2023-07-14T11:30:35.158 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/xcbc.ko 2023-07-14T11:30:35.158 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/xor.ko 2023-07-14T11:30:35.158 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/xxhash_generic.ko 2023-07-14T11:30:35.158 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers 2023-07-14T11:30:35.158 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi 2023-07-14T11:30:35.159 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/acpi_ipmi.ko 2023-07-14T11:30:35.159 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/acpi_pad.ko 2023-07-14T11:30:35.159 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/custom_method.ko 2023-07-14T11:30:35.159 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/ec_sys.ko 2023-07-14T11:30:35.159 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/platform_profile.ko 2023-07-14T11:30:35.159 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/sbs.ko 2023-07-14T11:30:35.159 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/sbshc.ko 2023-07-14T11:30:35.159 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/video.ko 2023-07-14T11:30:35.160 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata 2023-07-14T11:30:35.160 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/acard-ahci.ko 2023-07-14T11:30:35.160 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/ahci_platform.ko 2023-07-14T11:30:35.160 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/ata_generic.ko 2023-07-14T11:30:35.160 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/libahci_platform.ko 2023-07-14T11:30:35.160 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_acpi.ko 2023-07-14T11:30:35.160 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ali.ko 2023-07-14T11:30:35.161 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_amd.ko 2023-07-14T11:30:35.161 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_artop.ko 2023-07-14T11:30:35.161 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_atiixp.ko 2023-07-14T11:30:35.161 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_atp867x.ko 2023-07-14T11:30:35.161 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_cmd640.ko 2023-07-14T11:30:35.161 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_cmd64x.ko 2023-07-14T11:30:35.161 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_cypress.ko 2023-07-14T11:30:35.162 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_efar.ko 2023-07-14T11:30:35.162 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt366.ko 2023-07-14T11:30:35.162 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt37x.ko 2023-07-14T11:30:35.162 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt3x2n.ko 2023-07-14T11:30:35.162 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt3x3.ko 2023-07-14T11:30:35.162 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_it8213.ko 2023-07-14T11:30:35.162 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_it821x.ko 2023-07-14T11:30:35.163 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_jmicron.ko 2023-07-14T11:30:35.163 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_marvell.ko 2023-07-14T11:30:35.163 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_mpiix.ko 2023-07-14T11:30:35.163 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_netcell.ko 2023-07-14T11:30:35.163 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ninja32.ko 2023-07-14T11:30:35.163 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ns87410.ko 2023-07-14T11:30:35.163 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ns87415.ko 2023-07-14T11:30:35.164 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_oldpiix.ko 2023-07-14T11:30:35.164 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_opti.ko 2023-07-14T11:30:35.164 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_optidma.ko 2023-07-14T11:30:35.164 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_pcmcia.ko 2023-07-14T11:30:35.164 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_pdc2027x.ko 2023-07-14T11:30:35.164 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-07-14T11:30:35.164 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_piccolo.ko 2023-07-14T11:30:35.165 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_rdc.ko 2023-07-14T11:30:35.165 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sch.ko 2023-07-14T11:30:35.165 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_serverworks.ko 2023-07-14T11:30:35.165 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sil680.ko 2023-07-14T11:30:35.166 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sis.ko 2023-07-14T11:30:35.166 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sl82c105.ko 2023-07-14T11:30:35.166 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_triflex.ko 2023-07-14T11:30:35.166 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_via.ko 2023-07-14T11:30:35.166 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pdc_adma.ko 2023-07-14T11:30:35.167 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_inic162x.ko 2023-07-14T11:30:35.167 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_mv.ko 2023-07-14T11:30:35.167 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_nv.ko 2023-07-14T11:30:35.167 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_promise.ko 2023-07-14T11:30:35.167 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_qstor.ko 2023-07-14T11:30:35.167 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sil.ko 2023-07-14T11:30:35.168 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sil24.ko 2023-07-14T11:30:35.168 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sis.ko 2023-07-14T11:30:35.168 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_svw.ko 2023-07-14T11:30:35.168 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sx4.ko 2023-07-14T11:30:35.168 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_uli.ko 2023-07-14T11:30:35.168 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_via.ko 2023-07-14T11:30:35.169 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_vsc.ko 2023-07-14T11:30:35.169 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/auxdisplay 2023-07-14T11:30:35.169 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/auxdisplay/ks0108.ko 2023-07-14T11:30:35.169 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base 2023-07-14T11:30:35.169 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base/regmap 2023-07-14T11:30:35.170 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base/regmap/regmap-i2c.ko 2023-07-14T11:30:35.171 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base/regmap/regmap-mmio.ko 2023-07-14T11:30:35.171 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/bcma 2023-07-14T11:30:35.171 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/bcma/bcma.ko 2023-07-14T11:30:35.171 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block 2023-07-14T11:30:35.171 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/aoe 2023-07-14T11:30:35.172 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/aoe/aoe.ko 2023-07-14T11:30:35.172 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/brd.ko 2023-07-14T11:30:35.172 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/drbd 2023-07-14T11:30:35.172 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/drbd/drbd.ko 2023-07-14T11:30:35.172 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/floppy.ko 2023-07-14T11:30:35.172 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/loop.ko 2023-07-14T11:30:35.173 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/mtip32xx 2023-07-14T11:30:35.173 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-07-14T11:30:35.173 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/nbd.ko 2023-07-14T11:30:35.173 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/null_blk 2023-07-14T11:30:35.173 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/null_blk/null_blk.ko 2023-07-14T11:30:35.174 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/pktcdvd.ko 2023-07-14T11:30:35.174 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/rbd.ko 2023-07-14T11:30:35.174 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/virtio_blk.ko 2023-07-14T11:30:35.174 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/xen-blkback 2023-07-14T11:30:35.174 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-07-14T11:30:35.174 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/xen-blkfront.ko 2023-07-14T11:30:35.174 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/zram 2023-07-14T11:30:35.175 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/zram/zram.ko 2023-07-14T11:30:35.175 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char 2023-07-14T11:30:35.175 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hangcheck-timer.ko 2023-07-14T11:30:35.175 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random 2023-07-14T11:30:35.175 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/amd-rng.ko 2023-07-14T11:30:35.175 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/intel-rng.ko 2023-07-14T11:30:35.176 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-07-14T11:30:35.176 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/via-rng.ko 2023-07-14T11:30:35.176 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/virtio-rng.ko 2023-07-14T11:30:35.176 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi 2023-07-14T11:30:35.176 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-07-14T11:30:35.177 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-07-14T11:30:35.177 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-07-14T11:30:35.177 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_si.ko 2023-07-14T11:30:35.177 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-07-14T11:30:35.177 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-07-14T11:30:35.177 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/lp.ko 2023-07-14T11:30:35.177 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/mwave 2023-07-14T11:30:35.178 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/mwave/mwave.ko 2023-07-14T11:30:35.178 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ppdev.ko 2023-07-14T11:30:35.178 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tlclk.ko 2023-07-14T11:30:35.178 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm 2023-07-14T11:30:35.178 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm.ko 2023-07-14T11:30:35.178 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_atmel.ko 2023-07-14T11:30:35.178 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_infineon.ko 2023-07-14T11:30:35.178 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_nsc.ko 2023-07-14T11:30:35.179 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_tis.ko 2023-07-14T11:30:35.179 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-07-14T11:30:35.179 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/uv_mmtimer.ko 2023-07-14T11:30:35.179 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/virtio_console.ko 2023-07-14T11:30:35.179 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq 2023-07-14T11:30:35.179 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-07-14T11:30:35.179 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-07-14T11:30:35.180 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/p4-clockmod.ko 2023-07-14T11:30:35.180 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-07-14T11:30:35.180 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/powernow-k8.ko 2023-07-14T11:30:35.180 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/speedstep-lib.ko 2023-07-14T11:30:35.180 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto 2023-07-14T11:30:35.180 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/ccp 2023-07-14T11:30:35.181 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-07-14T11:30:35.181 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/ccp/ccp.ko 2023-07-14T11:30:35.181 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel 2023-07-14T11:30:35.181 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat 2023-07-14T11:30:35.181 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_common 2023-07-14T11:30:35.181 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-07-14T11:30:35.182 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-07-14T11:30:35.182 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-07-14T11:30:35.182 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/padlock-aes.ko 2023-07-14T11:30:35.182 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/padlock-sha.ko 2023-07-14T11:30:35.182 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/virtio 2023-07-14T11:30:35.182 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-07-14T11:30:35.183 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dca 2023-07-14T11:30:35.183 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dca/dca.ko 2023-07-14T11:30:35.183 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/devfreq 2023-07-14T11:30:35.183 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-07-14T11:30:35.183 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma 2023-07-14T11:30:35.183 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/dw 2023-07-14T11:30:35.183 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/dw/dw_dmac.ko 2023-07-14T11:30:35.184 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/ioat 2023-07-14T11:30:35.184 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/ioat/ioatdma.ko 2023-07-14T11:30:35.184 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac 2023-07-14T11:30:35.184 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/amd64_edac.ko 2023-07-14T11:30:35.184 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/e752x_edac.ko 2023-07-14T11:30:35.184 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/edac_mce_amd.ko 2023-07-14T11:30:35.184 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i3000_edac.ko 2023-07-14T11:30:35.185 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i3200_edac.ko 2023-07-14T11:30:35.185 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i5100_edac.ko 2023-07-14T11:30:35.185 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i5400_edac.ko 2023-07-14T11:30:35.185 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i7300_edac.ko 2023-07-14T11:30:35.185 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i7core_edac.ko 2023-07-14T11:30:35.185 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i82975x_edac.ko 2023-07-14T11:30:35.185 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/ie31200_edac.ko 2023-07-14T11:30:35.185 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/sb_edac.ko 2023-07-14T11:30:35.186 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/x38_edac.ko 2023-07-14T11:30:35.186 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/firmware 2023-07-14T11:30:35.186 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/firmware/edd.ko 2023-07-14T11:30:35.186 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/firmware/iscsi_ibft.ko 2023-07-14T11:30:35.186 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio 2023-07-14T11:30:35.186 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-ich.ko 2023-07-14T11:30:35.186 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-viperboard.ko 2023-07-14T11:30:35.187 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid 2023-07-14T11:30:35.187 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-appleir.ko 2023-07-14T11:30:35.187 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-aureal.ko 2023-07-14T11:30:35.187 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-axff.ko 2023-07-14T11:30:35.187 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-dr.ko 2023-07-14T11:30:35.187 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-elecom.ko 2023-07-14T11:30:35.188 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-elo.ko 2023-07-14T11:30:35.188 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-emsff.ko 2023-07-14T11:30:35.188 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-gaff.ko 2023-07-14T11:30:35.188 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-gt683r.ko 2023-07-14T11:30:35.188 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-gyration.ko 2023-07-14T11:30:35.188 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-holtek-kbd.ko 2023-07-14T11:30:35.188 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-holtek-mouse.ko 2023-07-14T11:30:35.189 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-holtekff.ko 2023-07-14T11:30:35.189 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-hyperv.ko 2023-07-14T11:30:35.189 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-icade.ko 2023-07-14T11:30:35.189 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-keytouch.ko 2023-07-14T11:30:35.189 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-kye.ko 2023-07-14T11:30:35.189 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-lcpower.ko 2023-07-14T11:30:35.189 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-led.ko 2023-07-14T11:30:35.190 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-lenovo.ko 2023-07-14T11:30:35.190 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-logitech-dj.ko 2023-07-14T11:30:35.190 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-07-14T11:30:35.190 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-multitouch.ko 2023-07-14T11:30:35.190 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-ortek.ko 2023-07-14T11:30:35.190 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-penmount.ko 2023-07-14T11:30:35.190 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-petalynx.ko 2023-07-14T11:30:35.191 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-picolcd.ko 2023-07-14T11:30:35.191 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-pl.ko 2023-07-14T11:30:35.191 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-plantronics.ko 2023-07-14T11:30:35.191 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-primax.ko 2023-07-14T11:30:35.191 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-rmi.ko 2023-07-14T11:30:35.191 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-arvo.ko 2023-07-14T11:30:35.191 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-common.ko 2023-07-14T11:30:35.192 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-isku.ko 2023-07-14T11:30:35.192 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-kone.ko 2023-07-14T11:30:35.192 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-07-14T11:30:35.192 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-konepure.ko 2023-07-14T11:30:35.192 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-07-14T11:30:35.192 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-lua.ko 2023-07-14T11:30:35.192 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-pyra.ko 2023-07-14T11:30:35.193 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-ryos.ko 2023-07-14T11:30:35.193 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-savu.ko 2023-07-14T11:30:35.193 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat.ko 2023-07-14T11:30:35.193 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-saitek.ko 2023-07-14T11:30:35.193 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-samsung.ko 2023-07-14T11:30:35.193 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sensor-hub.ko 2023-07-14T11:30:35.193 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sjoy.ko 2023-07-14T11:30:35.194 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sony.ko 2023-07-14T11:30:35.194 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-speedlink.ko 2023-07-14T11:30:35.194 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-steelseries.ko 2023-07-14T11:30:35.194 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sunplus.ko 2023-07-14T11:30:35.194 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-thrustmaster.ko 2023-07-14T11:30:35.194 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-tivo.ko 2023-07-14T11:30:35.195 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-tmff.ko 2023-07-14T11:30:35.195 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-topseed.ko 2023-07-14T11:30:35.195 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-twinhan.ko 2023-07-14T11:30:35.195 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-uclogic.ko 2023-07-14T11:30:35.195 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-waltop.ko 2023-07-14T11:30:35.195 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-wiimote.ko 2023-07-14T11:30:35.196 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-xinmo.ko 2023-07-14T11:30:35.196 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-zpff.ko 2023-07-14T11:30:35.196 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-zydacron.ko 2023-07-14T11:30:35.196 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/uhid.ko 2023-07-14T11:30:35.196 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/wacom.ko 2023-07-14T11:30:35.197 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv 2023-07-14T11:30:35.198 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv/hv_balloon.ko 2023-07-14T11:30:35.198 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv/hv_utils.ko 2023-07-14T11:30:35.199 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv/hv_vmbus.ko 2023-07-14T11:30:35.199 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon 2023-07-14T11:30:35.199 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/abituguru.ko 2023-07-14T11:30:35.199 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/abituguru3.ko 2023-07-14T11:30:35.200 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/acpi_power_meter.ko 2023-07-14T11:30:35.200 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ad7414.ko 2023-07-14T11:30:35.200 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ad7418.ko 2023-07-14T11:30:35.201 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adc128d818.ko 2023-07-14T11:30:35.201 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1025.ko 2023-07-14T11:30:35.201 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1026.ko 2023-07-14T11:30:35.202 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1029.ko 2023-07-14T11:30:35.202 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1031.ko 2023-07-14T11:30:35.202 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm9240.ko 2023-07-14T11:30:35.203 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ads7828.ko 2023-07-14T11:30:35.203 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7410.ko 2023-07-14T11:30:35.204 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7411.ko 2023-07-14T11:30:35.204 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7462.ko 2023-07-14T11:30:35.205 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7470.ko 2023-07-14T11:30:35.205 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7475.ko 2023-07-14T11:30:35.206 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7x10.ko 2023-07-14T11:30:35.206 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/amc6821.ko 2023-07-14T11:30:35.206 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/applesmc.ko 2023-07-14T11:30:35.207 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/asb100.ko 2023-07-14T11:30:35.207 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/asc7621.ko 2023-07-14T11:30:35.207 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/asus_atk0110.ko 2023-07-14T11:30:35.208 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/atxp1.ko 2023-07-14T11:30:35.209 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/coretemp.ko 2023-07-14T11:30:35.209 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-07-14T11:30:35.210 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/dme1737.ko 2023-07-14T11:30:35.211 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ds1621.ko 2023-07-14T11:30:35.211 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ds620.ko 2023-07-14T11:30:35.212 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/emc1403.ko 2023-07-14T11:30:35.213 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/emc6w201.ko 2023-07-14T11:30:35.214 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/f71805f.ko 2023-07-14T11:30:35.215 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/f71882fg.ko 2023-07-14T11:30:35.216 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/f75375s.ko 2023-07-14T11:30:35.216 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/fam15h_power.ko 2023-07-14T11:30:35.217 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/fschmd.ko 2023-07-14T11:30:35.219 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/g760a.ko 2023-07-14T11:30:35.220 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/g762.ko 2023-07-14T11:30:35.220 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/gl518sm.ko 2023-07-14T11:30:35.222 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/gl520sm.ko 2023-07-14T11:30:35.222 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/hwmon-vid.ko 2023-07-14T11:30:35.223 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/i5500_temp.ko 2023-07-14T11:30:35.223 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/i5k_amb.ko 2023-07-14T11:30:35.224 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ibmaem.ko 2023-07-14T11:30:35.224 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ibmpex.ko 2023-07-14T11:30:35.225 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ina209.ko 2023-07-14T11:30:35.225 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ina2xx.ko 2023-07-14T11:30:35.225 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/it87.ko 2023-07-14T11:30:35.226 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/k10temp.ko 2023-07-14T11:30:35.226 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/k8temp.ko 2023-07-14T11:30:35.226 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lineage-pem.ko 2023-07-14T11:30:35.226 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm63.ko 2023-07-14T11:30:35.227 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm73.ko 2023-07-14T11:30:35.227 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm75.ko 2023-07-14T11:30:35.227 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm77.ko 2023-07-14T11:30:35.228 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm78.ko 2023-07-14T11:30:35.229 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm80.ko 2023-07-14T11:30:35.229 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm83.ko 2023-07-14T11:30:35.230 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm85.ko 2023-07-14T11:30:35.231 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm87.ko 2023-07-14T11:30:35.232 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm90.ko 2023-07-14T11:30:35.232 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm92.ko 2023-07-14T11:30:35.233 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm93.ko 2023-07-14T11:30:35.233 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm95234.ko 2023-07-14T11:30:35.233 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm95241.ko 2023-07-14T11:30:35.234 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm95245.ko 2023-07-14T11:30:35.234 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc2945.ko 2023-07-14T11:30:35.234 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4151.ko 2023-07-14T11:30:35.235 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4215.ko 2023-07-14T11:30:35.235 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4222.ko 2023-07-14T11:30:35.236 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4245.ko 2023-07-14T11:30:35.236 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4260.ko 2023-07-14T11:30:35.236 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4261.ko 2023-07-14T11:30:35.237 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max16065.ko 2023-07-14T11:30:35.237 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max1619.ko 2023-07-14T11:30:35.238 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max1668.ko 2023-07-14T11:30:35.238 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max197.ko 2023-07-14T11:30:35.238 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max6639.ko 2023-07-14T11:30:35.238 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max6650.ko 2023-07-14T11:30:35.238 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max6697.ko 2023-07-14T11:30:35.238 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/mcp3021.ko 2023-07-14T11:30:35.238 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct6683.ko 2023-07-14T11:30:35.239 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct6775-core.ko 2023-07-14T11:30:35.239 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct6775.ko 2023-07-14T11:30:35.239 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct7802.ko 2023-07-14T11:30:35.239 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pc87360.ko 2023-07-14T11:30:35.239 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pc87427.ko 2023-07-14T11:30:35.239 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pcf8591.ko 2023-07-14T11:30:35.239 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus 2023-07-14T11:30:35.240 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-07-14T11:30:35.240 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-07-14T11:30:35.240 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-07-14T11:30:35.240 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/max16064.ko 2023-07-14T11:30:35.240 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/max34440.ko 2023-07-14T11:30:35.240 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/max8688.ko 2023-07-14T11:30:35.240 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-07-14T11:30:35.240 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-07-14T11:30:35.241 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-07-14T11:30:35.241 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-07-14T11:30:35.241 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-07-14T11:30:35.241 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-07-14T11:30:35.241 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/powr1220.ko 2023-07-14T11:30:35.241 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sch5627.ko 2023-07-14T11:30:35.241 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sch5636.ko 2023-07-14T11:30:35.519 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sch56xx-common.ko 2023-07-14T11:30:35.519 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sht15.ko 2023-07-14T11:30:35.519 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sht21.ko 2023-07-14T11:30:35.519 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/shtc1.ko 2023-07-14T11:30:35.520 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sis5595.ko 2023-07-14T11:30:35.520 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/smsc47b397.ko 2023-07-14T11:30:35.520 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/smsc47m1.ko 2023-07-14T11:30:35.520 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/smsc47m192.ko 2023-07-14T11:30:35.520 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/thmc50.ko 2023-07-14T11:30:35.520 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp102.ko 2023-07-14T11:30:35.520 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp103.ko 2023-07-14T11:30:35.521 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp401.ko 2023-07-14T11:30:35.521 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp421.ko 2023-07-14T11:30:35.521 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/via-cputemp.ko 2023-07-14T11:30:35.521 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/via686a.ko 2023-07-14T11:30:35.521 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/vt1211.ko 2023-07-14T11:30:35.521 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/vt8231.ko 2023-07-14T11:30:35.521 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83627ehf.ko 2023-07-14T11:30:35.522 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83627hf.ko 2023-07-14T11:30:35.522 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83781d.ko 2023-07-14T11:30:35.522 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83791d.ko 2023-07-14T11:30:35.522 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83792d.ko 2023-07-14T11:30:35.522 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83793.ko 2023-07-14T11:30:35.522 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83795.ko 2023-07-14T11:30:35.522 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83l785ts.ko 2023-07-14T11:30:35.522 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83l786ng.ko 2023-07-14T11:30:35.523 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c 2023-07-14T11:30:35.523 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/algos 2023-07-14T11:30:35.523 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-07-14T11:30:35.523 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-07-14T11:30:35.523 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses 2023-07-14T11:30:35.523 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-07-14T11:30:35.523 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-07-14T11:30:35.523 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-07-14T11:30:35.524 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-07-14T11:30:35.524 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-07-14T11:30:35.524 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-07-14T11:30:35.524 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-07-14T11:30:35.524 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-07-14T11:30:35.524 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-i801.ko 2023-07-14T11:30:35.524 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-isch.ko 2023-07-14T11:30:35.525 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-07-14T11:30:35.525 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-07-14T11:30:35.525 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-07-14T11:30:35.525 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-parport.ko 2023-07-14T11:30:35.525 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-07-14T11:30:35.525 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-07-14T11:30:35.525 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-07-14T11:30:35.525 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-07-14T11:30:35.526 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-07-14T11:30:35.526 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-07-14T11:30:35.526 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-via.ko 2023-07-14T11:30:35.526 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-07-14T11:30:35.526 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-07-14T11:30:35.526 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-dev.ko 2023-07-14T11:30:35.526 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-mux.ko 2023-07-14T11:30:35.527 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-07-14T11:30:35.527 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-smbus.ko 2023-07-14T11:30:35.527 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-stub.ko 2023-07-14T11:30:35.527 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband 2023-07-14T11:30:35.527 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core 2023-07-14T11:30:35.527 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_cm.ko 2023-07-14T11:30:35.527 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_core.ko 2023-07-14T11:30:35.527 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_umad.ko 2023-07-14T11:30:35.528 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-07-14T11:30:35.528 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/iw_cm.ko 2023-07-14T11:30:35.528 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/rdma_cm.ko 2023-07-14T11:30:35.528 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-07-14T11:30:35.528 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw 2023-07-14T11:30:35.528 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/cxgb4 2023-07-14T11:30:35.528 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-07-14T11:30:35.528 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx4 2023-07-14T11:30:35.529 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-07-14T11:30:35.529 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx5 2023-07-14T11:30:35.529 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-07-14T11:30:35.529 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mthca 2023-07-14T11:30:35.529 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-07-14T11:30:35.529 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp 2023-07-14T11:30:35.529 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/ipoib 2023-07-14T11:30:35.530 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-07-14T11:30:35.530 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/iser 2023-07-14T11:30:35.530 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-07-14T11:30:35.530 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/isert 2023-07-14T11:30:35.530 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-07-14T11:30:35.530 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srp 2023-07-14T11:30:35.530 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-07-14T11:30:35.530 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srpt 2023-07-14T11:30:35.531 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-07-14T11:30:35.531 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input 2023-07-14T11:30:35.531 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport 2023-07-14T11:30:35.531 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-07-14T11:30:35.531 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/fm801-gp.ko 2023-07-14T11:30:35.531 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/gameport.ko 2023-07-14T11:30:35.531 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/lightning.ko 2023-07-14T11:30:35.531 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/ns558.ko 2023-07-14T11:30:35.532 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/joydev.ko 2023-07-14T11:30:35.532 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard 2023-07-14T11:30:35.532 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/gpio_keys.ko 2023-07-14T11:30:35.532 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse 2023-07-14T11:30:35.532 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/appletouch.ko 2023-07-14T11:30:35.532 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/bcm5974.ko 2023-07-14T11:30:35.533 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/cyapatp.ko 2023-07-14T11:30:35.533 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/elan_i2c.ko 2023-07-14T11:30:35.533 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/sermouse.ko 2023-07-14T11:30:35.533 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-07-14T11:30:35.533 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/synaptics_usb.ko 2023-07-14T11:30:35.533 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/vsxxxaa.ko 2023-07-14T11:30:35.533 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/rmi4 2023-07-14T11:30:35.534 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/rmi4/rmi_core.ko 2023-07-14T11:30:35.534 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio 2023-07-14T11:30:35.534 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/altera_ps2.ko 2023-07-14T11:30:35.534 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/arc_ps2.ko 2023-07-14T11:30:35.534 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-07-14T11:30:35.534 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/serio_raw.ko 2023-07-14T11:30:35.534 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/sparse-keymap.ko 2023-07-14T11:30:35.534 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/iommu 2023-07-14T11:30:35.535 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/iommu/amd 2023-07-14T11:30:35.535 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/iommu/amd/iommu_v2.ko 2023-07-14T11:30:35.535 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds 2023-07-14T11:30:35.535 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-blinkm.ko 2023-07-14T11:30:35.535 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-lm3530.ko 2023-07-14T11:30:35.535 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-lp3944.ko 2023-07-14T11:30:35.535 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-ss4200.ko 2023-07-14T11:30:35.535 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger 2023-07-14T11:30:35.536 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-07-14T11:30:35.536 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-07-14T11:30:35.536 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-07-14T11:30:35.536 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-07-14T11:30:35.536 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-07-14T11:30:35.536 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-07-14T11:30:35.536 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-07-14T11:30:35.537 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-07-14T11:30:35.537 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md 2023-07-14T11:30:35.537 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/bcache 2023-07-14T11:30:35.537 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/bcache/bcache.ko 2023-07-14T11:30:35.537 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-bio-prison.ko 2023-07-14T11:30:35.537 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-cache-smq.ko 2023-07-14T11:30:35.537 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-cache.ko 2023-07-14T11:30:35.537 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-crypt.ko 2023-07-14T11:30:35.538 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-delay.ko 2023-07-14T11:30:35.538 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-flakey.ko 2023-07-14T11:30:35.538 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-log-userspace.ko 2023-07-14T11:30:35.538 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-multipath.ko 2023-07-14T11:30:35.538 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-queue-length.ko 2023-07-14T11:30:35.538 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-raid.ko 2023-07-14T11:30:35.538 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-round-robin.ko 2023-07-14T11:30:35.538 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-service-time.ko 2023-07-14T11:30:35.539 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-switch.ko 2023-07-14T11:30:35.539 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-thin-pool.ko 2023-07-14T11:30:35.539 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-verity.ko 2023-07-14T11:30:35.539 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/faulty.ko 2023-07-14T11:30:35.539 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/linear.ko 2023-07-14T11:30:35.539 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/multipath.ko 2023-07-14T11:30:35.539 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/persistent-data 2023-07-14T11:30:35.540 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-07-14T11:30:35.540 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid0.ko 2023-07-14T11:30:35.540 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid1.ko 2023-07-14T11:30:35.540 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid10.ko 2023-07-14T11:30:35.540 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid456.ko 2023-07-14T11:30:35.540 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message 2023-07-14T11:30:35.540 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion 2023-07-14T11:30:35.540 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptbase.ko 2023-07-14T11:30:35.541 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptctl.ko 2023-07-14T11:30:35.541 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptfc.ko 2023-07-14T11:30:35.541 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptlan.ko 2023-07-14T11:30:35.541 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptsas.ko 2023-07-14T11:30:35.541 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptscsih.ko 2023-07-14T11:30:35.541 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptspi.ko 2023-07-14T11:30:35.541 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd 2023-07-14T11:30:35.541 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/lpc_ich.ko 2023-07-14T11:30:35.542 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/lpc_sch.ko 2023-07-14T11:30:35.542 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/mfd-core.ko 2023-07-14T11:30:35.542 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/sm501.ko 2023-07-14T11:30:35.542 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/viperboard.ko 2023-07-14T11:30:35.542 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/vx855.ko 2023-07-14T11:30:35.542 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/wl1273-core.ko 2023-07-14T11:30:35.542 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc 2023-07-14T11:30:35.543 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/altera-stapl 2023-07-14T11:30:35.543 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-07-14T11:30:35.543 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/apds9802als.ko 2023-07-14T11:30:35.543 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/apds990x.ko 2023-07-14T11:30:35.543 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/bh1770glc.ko 2023-07-14T11:30:35.543 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/cb710 2023-07-14T11:30:35.543 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/cb710/cb710.ko 2023-07-14T11:30:35.543 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom 2023-07-14T11:30:35.544 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/at24.ko 2023-07-14T11:30:35.544 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/eeprom.ko 2023-07-14T11:30:35.544 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-07-14T11:30:35.544 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/max6875.ko 2023-07-14T11:30:35.544 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/enclosure.ko 2023-07-14T11:30:35.544 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/hpilo.ko 2023-07-14T11:30:35.544 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/isl29003.ko 2023-07-14T11:30:35.544 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/isl29020.ko 2023-07-14T11:30:35.545 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/lis3lv02d 2023-07-14T11:30:35.545 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-07-14T11:30:35.545 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-07-14T11:30:35.545 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei 2023-07-14T11:30:35.545 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei/mei-me.ko 2023-07-14T11:30:35.545 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei/mei-txe.ko 2023-07-14T11:30:35.545 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei/mei.ko 2023-07-14T11:30:35.545 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-gru 2023-07-14T11:30:35.546 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-gru/gru.ko 2023-07-14T11:30:35.546 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp 2023-07-14T11:30:35.546 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp/xp.ko 2023-07-14T11:30:35.546 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp/xpc.ko 2023-07-14T11:30:35.546 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-07-14T11:30:35.546 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/tifm_7xx1.ko 2023-07-14T11:30:35.546 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/tifm_core.ko 2023-07-14T11:30:35.546 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/tsl2550.ko 2023-07-14T11:30:35.547 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/vmw_balloon.ko 2023-07-14T11:30:35.547 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/vmw_vmci 2023-07-14T11:30:35.547 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-07-14T11:30:35.547 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net 2023-07-14T11:30:35.547 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/bonding 2023-07-14T11:30:35.547 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/bonding/bonding.ko 2023-07-14T11:30:35.547 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa 2023-07-14T11:30:35.548 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/b53 2023-07-14T11:30:35.548 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/b53/b53_common.ko 2023-07-14T11:30:35.548 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/bcm-sf2.ko 2023-07-14T11:30:35.548 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/mv88e6060.ko 2023-07-14T11:30:35.548 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/mv88e6xxx 2023-07-14T11:30:35.548 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-07-14T11:30:35.548 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dummy.ko 2023-07-14T11:30:35.548 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/eql.ko 2023-07-14T11:30:35.549 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet 2023-07-14T11:30:35.549 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com 2023-07-14T11:30:35.549 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-07-14T11:30:35.549 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-07-14T11:30:35.549 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-07-14T11:30:35.549 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-07-14T11:30:35.549 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390 2023-07-14T11:30:35.550 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/8390.ko 2023-07-14T11:30:35.550 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-07-14T11:30:35.550 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-07-14T11:30:35.550 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-07-14T11:30:35.550 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/adaptec 2023-07-14T11:30:35.550 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-07-14T11:30:35.550 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/agere 2023-07-14T11:30:35.550 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/agere/et131x.ko 2023-07-14T11:30:35.551 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/alteon 2023-07-14T11:30:35.551 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-07-14T11:30:35.551 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/altera 2023-07-14T11:30:35.551 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-07-14T11:30:35.551 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd 2023-07-14T11:30:35.551 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-07-14T11:30:35.551 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-07-14T11:30:35.551 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-07-14T11:30:35.552 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros 2023-07-14T11:30:35.552 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/alx 2023-07-14T11:30:35.552 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-07-14T11:30:35.552 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1c 2023-07-14T11:30:35.552 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-07-14T11:30:35.552 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1e 2023-07-14T11:30:35.552 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-07-14T11:30:35.552 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atlx 2023-07-14T11:30:35.553 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-07-14T11:30:35.553 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-07-14T11:30:35.553 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom 2023-07-14T11:30:35.553 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-07-14T11:30:35.553 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-07-14T11:30:35.553 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-07-14T11:30:35.553 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-07-14T11:30:35.554 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-07-14T11:30:35.554 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/genet 2023-07-14T11:30:35.554 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-07-14T11:30:35.554 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-07-14T11:30:35.554 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/brocade 2023-07-14T11:30:35.554 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/brocade/bna 2023-07-14T11:30:35.554 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-07-14T11:30:35.554 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio 2023-07-14T11:30:35.555 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb 2023-07-14T11:30:35.555 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-07-14T11:30:35.555 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-07-14T11:30:35.555 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-07-14T11:30:35.555 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-07-14T11:30:35.555 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-07-14T11:30:35.555 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-07-14T11:30:35.555 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-07-14T11:30:35.556 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-07-14T11:30:35.556 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-07-14T11:30:35.556 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/cisco 2023-07-14T11:30:35.556 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/cisco/enic 2023-07-14T11:30:35.556 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-07-14T11:30:35.557 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec 2023-07-14T11:30:35.557 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip 2023-07-14T11:30:35.557 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-07-14T11:30:35.557 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-07-14T11:30:35.557 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-07-14T11:30:35.558 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-07-14T11:30:35.558 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-07-14T11:30:35.558 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-07-14T11:30:35.558 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dlink 2023-07-14T11:30:35.558 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-07-14T11:30:35.558 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-07-14T11:30:35.558 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dnet.ko 2023-07-14T11:30:35.559 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/emulex 2023-07-14T11:30:35.559 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/emulex/benet 2023-07-14T11:30:35.559 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-07-14T11:30:35.559 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/ethoc.ko 2023-07-14T11:30:35.559 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/fealnx.ko 2023-07-14T11:30:35.559 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel 2023-07-14T11:30:35.559 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e100.ko 2023-07-14T11:30:35.559 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000 2023-07-14T11:30:35.560 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-07-14T11:30:35.560 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000e 2023-07-14T11:30:35.560 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-07-14T11:30:35.560 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/fm10k 2023-07-14T11:30:35.560 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-07-14T11:30:35.560 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/i40e 2023-07-14T11:30:35.560 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-07-14T11:30:35.560 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/iavf 2023-07-14T11:30:35.561 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-07-14T11:30:35.561 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igb 2023-07-14T11:30:35.561 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-07-14T11:30:35.561 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igbvf 2023-07-14T11:30:35.561 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-07-14T11:30:35.561 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbe 2023-07-14T11:30:35.561 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-07-14T11:30:35.562 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbevf 2023-07-14T11:30:35.562 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-07-14T11:30:35.562 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/jme.ko 2023-07-14T11:30:35.562 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell 2023-07-14T11:30:35.562 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-07-14T11:30:35.562 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell/skge.ko 2023-07-14T11:30:35.562 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-07-14T11:30:35.562 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox 2023-07-14T11:30:35.563 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx4 2023-07-14T11:30:35.563 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-07-14T11:30:35.563 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-07-14T11:30:35.563 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx5 2023-07-14T11:30:35.563 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-07-14T11:30:35.563 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-07-14T11:30:35.563 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/micrel 2023-07-14T11:30:35.563 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-07-14T11:30:35.564 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/myricom 2023-07-14T11:30:35.564 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/myricom/myri10ge 2023-07-14T11:30:35.564 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-07-14T11:30:35.564 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/natsemi 2023-07-14T11:30:35.564 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-07-14T11:30:35.564 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-07-14T11:30:35.564 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/neterion 2023-07-14T11:30:35.564 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-07-14T11:30:35.565 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/nvidia 2023-07-14T11:30:35.565 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-07-14T11:30:35.565 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/packetengines 2023-07-14T11:30:35.565 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-07-14T11:30:35.565 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-07-14T11:30:35.565 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic 2023-07-14T11:30:35.565 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/netxen 2023-07-14T11:30:35.565 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-07-14T11:30:35.566 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-07-14T11:30:35.566 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-07-14T11:30:35.566 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-07-14T11:30:35.566 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rdc 2023-07-14T11:30:35.566 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-07-14T11:30:35.566 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek 2023-07-14T11:30:35.566 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-07-14T11:30:35.566 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-07-14T11:30:35.567 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/atp.ko 2023-07-14T11:30:35.567 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-07-14T11:30:35.567 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rocker 2023-07-14T11:30:35.567 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-07-14T11:30:35.567 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sfc 2023-07-14T11:30:35.567 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-07-14T11:30:35.567 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/silan 2023-07-14T11:30:35.568 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-07-14T11:30:35.568 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sis 2023-07-14T11:30:35.568 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sis/sis190.ko 2023-07-14T11:30:35.568 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sis/sis900.ko 2023-07-14T11:30:35.568 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc 2023-07-14T11:30:35.568 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-07-14T11:30:35.568 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-07-14T11:30:35.568 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-07-14T11:30:35.569 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-07-14T11:30:35.569 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro 2023-07-14T11:30:35.569 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/stmmac 2023-07-14T11:30:35.569 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-07-14T11:30:35.569 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-07-14T11:30:35.569 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun 2023-07-14T11:30:35.569 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/cassini.ko 2023-07-14T11:30:35.570 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/niu.ko 2023-07-14T11:30:35.570 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/sungem.ko 2023-07-14T11:30:35.570 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-07-14T11:30:35.570 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/tehuti 2023-07-14T11:30:35.570 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-07-14T11:30:35.570 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/ti 2023-07-14T11:30:35.570 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/ti/tlan.ko 2023-07-14T11:30:35.570 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/via 2023-07-14T11:30:35.571 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-07-14T11:30:35.571 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-07-14T11:30:35.571 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/wiznet 2023-07-14T11:30:35.571 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-07-14T11:30:35.571 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-07-14T11:30:35.571 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/xircom 2023-07-14T11:30:35.571 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-07-14T11:30:35.571 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/geneve.ko 2023-07-14T11:30:35.572 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/hyperv 2023-07-14T11:30:35.572 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-07-14T11:30:35.572 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ieee802154 2023-07-14T11:30:35.572 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ieee802154/fakelb.ko 2023-07-14T11:30:35.572 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ifb.ko 2023-07-14T11:30:35.572 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ipvlan 2023-07-14T11:30:35.572 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ipvlan/ipvlan.ko 2023-07-14T11:30:35.572 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/macvlan.ko 2023-07-14T11:30:35.573 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/macvtap.ko 2023-07-14T11:30:35.573 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio 2023-07-14T11:30:35.573 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio.ko 2023-07-14T11:30:35.573 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-07-14T11:30:35.573 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-07-14T11:30:35.573 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio/mdio-regmap.ko 2023-07-14T11:30:35.573 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mii.ko 2023-07-14T11:30:35.574 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/net_failover.ko 2023-07-14T11:30:35.574 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/netconsole.ko 2023-07-14T11:30:35.574 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/nlmon.ko 2023-07-14T11:30:35.574 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/pcs 2023-07-14T11:30:35.574 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/pcs/pcs-lynx.ko 2023-07-14T11:30:35.574 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-07-14T11:30:35.574 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy 2023-07-14T11:30:35.574 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/amd.ko 2023-07-14T11:30:35.575 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-07-14T11:30:35.575 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-07-14T11:30:35.575 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm7xxx.ko 2023-07-14T11:30:35.575 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm87xx.ko 2023-07-14T11:30:35.575 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/broadcom.ko 2023-07-14T11:30:35.575 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/cicada.ko 2023-07-14T11:30:35.575 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/davicom.ko 2023-07-14T11:30:35.575 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/dp83640.ko 2023-07-14T11:30:35.576 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/et1011c.ko 2023-07-14T11:30:35.576 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/icplus.ko 2023-07-14T11:30:35.576 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/lxt.ko 2023-07-14T11:30:35.576 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/marvell.ko 2023-07-14T11:30:35.576 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/micrel.ko 2023-07-14T11:30:35.576 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/national.ko 2023-07-14T11:30:35.576 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/phylink.ko 2023-07-14T11:30:35.576 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/qsemi.ko 2023-07-14T11:30:35.577 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/realtek.ko 2023-07-14T11:30:35.577 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/smsc.ko 2023-07-14T11:30:35.577 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/ste10Xp.ko 2023-07-14T11:30:35.577 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/vitesse.ko 2023-07-14T11:30:35.577 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp 2023-07-14T11:30:35.577 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/bsd_comp.ko 2023-07-14T11:30:35.577 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_async.ko 2023-07-14T11:30:35.578 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_deflate.ko 2023-07-14T11:30:35.578 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_generic.ko 2023-07-14T11:30:35.578 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_mppe.ko 2023-07-14T11:30:35.578 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_synctty.ko 2023-07-14T11:30:35.578 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/pppoe.ko 2023-07-14T11:30:35.578 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/pppox.ko 2023-07-14T11:30:35.578 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/pptp.ko 2023-07-14T11:30:35.578 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/slip 2023-07-14T11:30:35.579 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/slip/slhc.ko 2023-07-14T11:30:35.579 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/slip/slip.ko 2023-07-14T11:30:35.579 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/sungem_phy.ko 2023-07-14T11:30:35.579 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/tap.ko 2023-07-14T11:30:35.579 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team 2023-07-14T11:30:35.579 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team.ko 2023-07-14T11:30:35.579 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_activebackup.ko 2023-07-14T11:30:35.579 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_broadcast.ko 2023-07-14T11:30:35.580 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-07-14T11:30:35.580 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_random.ko 2023-07-14T11:30:35.580 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-07-14T11:30:35.580 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/tun.ko 2023-07-14T11:30:35.580 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/veth.ko 2023-07-14T11:30:35.580 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/virtio_net.ko 2023-07-14T11:30:35.580 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vmxnet3 2023-07-14T11:30:35.580 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-07-14T11:30:35.581 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vxlan 2023-07-14T11:30:35.581 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vxlan/vxlan.ko 2023-07-14T11:30:35.581 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/xen-netback 2023-07-14T11:30:35.581 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/xen-netback/xen-netback.ko 2023-07-14T11:30:35.581 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/xen-netfront.ko 2023-07-14T11:30:35.581 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ntb 2023-07-14T11:30:35.581 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ntb/ntb.ko 2023-07-14T11:30:35.581 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme 2023-07-14T11:30:35.582 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme/host 2023-07-14T11:30:35.582 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme/host/nvme-core.ko 2023-07-14T11:30:35.582 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme/host/nvme.ko 2023-07-14T11:30:35.582 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport 2023-07-14T11:30:35.582 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport.ko 2023-07-14T11:30:35.582 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport_cs.ko 2023-07-14T11:30:35.582 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport_pc.ko 2023-07-14T11:30:35.583 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport_serial.ko 2023-07-14T11:30:35.583 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci 2023-07-14T11:30:35.583 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/hotplug 2023-07-14T11:30:35.583 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-07-14T11:30:35.583 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/pcie 2023-07-14T11:30:35.583 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/pcie/aer_inject.ko 2023-07-14T11:30:35.583 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/xen-pcifront.ko 2023-07-14T11:30:35.583 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia 2023-07-14T11:30:35.584 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/i82092.ko 2023-07-14T11:30:35.584 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/pd6729.ko 2023-07-14T11:30:35.584 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/yenta_socket.ko 2023-07-14T11:30:35.584 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pinctrl 2023-07-14T11:30:35.584 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pinctrl/intel 2023-07-14T11:30:35.584 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-07-14T11:30:35.584 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform 2023-07-14T11:30:35.584 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86 2023-07-14T11:30:35.585 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/acer-wmi.ko 2023-07-14T11:30:35.585 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/acerhdf.ko 2023-07-14T11:30:35.585 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/apple-gmux.ko 2023-07-14T11:30:35.585 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/asus-laptop.ko 2023-07-14T11:30:35.585 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-07-14T11:30:35.585 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/asus-wmi.ko 2023-07-14T11:30:35.585 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/classmate-laptop.ko 2023-07-14T11:30:35.585 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-07-14T11:30:35.586 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-07-14T11:30:35.586 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-07-14T11:30:35.586 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-07-14T11:30:35.586 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/hdaps.ko 2023-07-14T11:30:35.586 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/intel 2023-07-14T11:30:35.586 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-07-14T11:30:35.586 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/intel_ips.ko 2023-07-14T11:30:35.586 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/msi-wmi.ko 2023-07-14T11:30:35.587 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/mxm-wmi.ko 2023-07-14T11:30:35.587 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-07-14T11:30:35.587 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/samsung-laptop.ko 2023-07-14T11:30:35.587 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/samsung-q10.ko 2023-07-14T11:30:35.587 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/topstar-laptop.ko 2023-07-14T11:30:35.587 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-07-14T11:30:35.587 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/toshiba_haps.ko 2023-07-14T11:30:35.587 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/wmi-bmof.ko 2023-07-14T11:30:35.588 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/wmi.ko 2023-07-14T11:30:35.588 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/powercap 2023-07-14T11:30:35.588 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/powercap/intel_rapl_common.ko 2023-07-14T11:30:35.588 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/powercap/intel_rapl_msr.ko 2023-07-14T11:30:35.588 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps 2023-07-14T11:30:35.588 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients 2023-07-14T11:30:35.588 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients/pps-gpio.ko 2023-07-14T11:30:35.589 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients/pps-ldisc.ko 2023-07-14T11:30:35.589 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients/pps_parport.ko 2023-07-14T11:30:35.589 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/pps_core.ko 2023-07-14T11:30:35.589 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ptp 2023-07-14T11:30:35.589 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ptp/ptp.ko 2023-07-14T11:30:35.589 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ptp/ptp_kvm.ko 2023-07-14T11:30:35.589 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc 2023-07-14T11:30:35.589 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-bq32k.ko 2023-07-14T11:30:35.590 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-bq4802.ko 2023-07-14T11:30:35.590 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1286.ko 2023-07-14T11:30:35.590 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1307.ko 2023-07-14T11:30:35.590 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1374.ko 2023-07-14T11:30:35.590 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1511.ko 2023-07-14T11:30:35.590 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1553.ko 2023-07-14T11:30:35.590 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1672.ko 2023-07-14T11:30:35.590 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1742.ko 2023-07-14T11:30:35.591 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds2404.ko 2023-07-14T11:30:35.591 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds3232.ko 2023-07-14T11:30:35.591 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-em3027.ko 2023-07-14T11:30:35.591 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-fm3130.ko 2023-07-14T11:30:35.591 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-isl12022.ko 2023-07-14T11:30:35.591 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-isl1208.ko 2023-07-14T11:30:35.591 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-m41t80.ko 2023-07-14T11:30:35.591 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-m48t35.ko 2023-07-14T11:30:35.592 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-m48t59.ko 2023-07-14T11:30:35.592 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-max6900.ko 2023-07-14T11:30:35.592 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-msm6242.ko 2023-07-14T11:30:35.592 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf2127.ko 2023-07-14T11:30:35.592 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf85063.ko 2023-07-14T11:30:35.592 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf8523.ko 2023-07-14T11:30:35.592 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf8563.ko 2023-07-14T11:30:35.592 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf8583.ko 2023-07-14T11:30:35.593 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rp5c01.ko 2023-07-14T11:30:35.593 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rs5c372.ko 2023-07-14T11:30:35.593 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-07-14T11:30:35.593 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rx8025.ko 2023-07-14T11:30:35.593 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rx8581.ko 2023-07-14T11:30:35.593 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-07-14T11:30:35.593 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-x1205.ko 2023-07-14T11:30:35.593 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi 2023-07-14T11:30:35.594 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/3w-9xxx.ko 2023-07-14T11:30:35.594 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/3w-sas.ko 2023-07-14T11:30:35.594 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/3w-xxxx.ko 2023-07-14T11:30:35.594 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/BusLogic.ko 2023-07-14T11:30:35.594 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/a100u2w.ko 2023-07-14T11:30:35.594 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aacraid 2023-07-14T11:30:35.594 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aacraid/aacraid.ko 2023-07-14T11:30:35.594 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/advansys.ko 2023-07-14T11:30:35.595 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aic7xxx 2023-07-14T11:30:35.595 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-07-14T11:30:35.595 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-07-14T11:30:35.595 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/am53c974.ko 2023-07-14T11:30:35.595 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/arcmsr 2023-07-14T11:30:35.595 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-07-14T11:30:35.595 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/atp870u.ko 2023-07-14T11:30:35.595 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/be2iscsi 2023-07-14T11:30:35.596 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-07-14T11:30:35.596 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bfa 2023-07-14T11:30:35.596 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bfa/bfa.ko 2023-07-14T11:30:35.596 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2fc 2023-07-14T11:30:35.596 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-07-14T11:30:35.596 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2i 2023-07-14T11:30:35.596 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-07-14T11:30:35.597 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ch.ko 2023-07-14T11:30:35.597 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/csiostor 2023-07-14T11:30:35.597 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/csiostor/csiostor.ko 2023-07-14T11:30:35.597 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi 2023-07-14T11:30:35.597 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb3i 2023-07-14T11:30:35.597 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-07-14T11:30:35.597 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb4i 2023-07-14T11:30:35.597 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-07-14T11:30:35.598 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-07-14T11:30:35.598 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/dc395x.ko 2023-07-14T11:30:35.598 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler 2023-07-14T11:30:35.598 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-07-14T11:30:35.598 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-07-14T11:30:35.598 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-07-14T11:30:35.598 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-07-14T11:30:35.598 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/dmx3191d.ko 2023-07-14T11:30:35.599 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/esas2r 2023-07-14T11:30:35.599 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/esas2r/esas2r.ko 2023-07-14T11:30:35.599 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/esp_scsi.ko 2023-07-14T11:30:35.599 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fcoe 2023-07-14T11:30:35.599 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fcoe/fcoe.ko 2023-07-14T11:30:35.599 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-07-14T11:30:35.599 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fnic 2023-07-14T11:30:35.599 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fnic/fnic.ko 2023-07-14T11:30:35.600 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/hpsa.ko 2023-07-14T11:30:35.600 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/hptiop.ko 2023-07-14T11:30:35.600 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/hv_storvsc.ko 2023-07-14T11:30:35.600 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/initio.ko 2023-07-14T11:30:35.600 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ipr.ko 2023-07-14T11:30:35.600 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ips.ko 2023-07-14T11:30:35.600 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/isci 2023-07-14T11:30:35.601 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/isci/isci.ko 2023-07-14T11:30:35.601 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-07-14T11:30:35.601 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/iscsi_tcp.ko 2023-07-14T11:30:35.601 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libfc 2023-07-14T11:30:35.601 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libfc/libfc.ko 2023-07-14T11:30:35.601 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libiscsi.ko 2023-07-14T11:30:35.601 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libiscsi_tcp.ko 2023-07-14T11:30:35.601 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libsas 2023-07-14T11:30:35.602 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libsas/libsas.ko 2023-07-14T11:30:35.602 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/lpfc 2023-07-14T11:30:35.602 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/lpfc/lpfc.ko 2023-07-14T11:30:35.602 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid 2023-07-14T11:30:35.602 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid.ko 2023-07-14T11:30:35.602 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-07-14T11:30:35.602 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-07-14T11:30:35.602 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-07-14T11:30:35.603 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mpt3sas 2023-07-14T11:30:35.603 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-07-14T11:30:35.603 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mvsas 2023-07-14T11:30:35.603 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mvsas/mvsas.ko 2023-07-14T11:30:35.603 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mvumi.ko 2023-07-14T11:30:35.603 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pm8001 2023-07-14T11:30:35.603 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-07-14T11:30:35.604 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pmcraid.ko 2023-07-14T11:30:35.604 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla1280.ko 2023-07-14T11:30:35.604 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla2xxx 2023-07-14T11:30:35.604 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-07-14T11:30:35.604 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-07-14T11:30:35.604 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla4xxx 2023-07-14T11:30:35.604 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-07-14T11:30:35.604 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/raid_class.ko 2023-07-14T11:30:35.605 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_debug.ko 2023-07-14T11:30:35.605 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_fc.ko 2023-07-14T11:30:35.605 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-07-14T11:30:35.605 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_sas.ko 2023-07-14T11:30:35.605 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_spi.ko 2023-07-14T11:30:35.605 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_srp.ko 2023-07-14T11:30:35.605 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ses.ko 2023-07-14T11:30:35.605 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/st.ko 2023-07-14T11:30:35.606 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/stex.ko 2023-07-14T11:30:35.606 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/sym53c8xx_2 2023-07-14T11:30:35.606 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-07-14T11:30:35.606 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/virtio_scsi.ko 2023-07-14T11:30:35.606 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/vmw_pvscsi.ko 2023-07-14T11:30:35.606 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/wd719x.ko 2023-07-14T11:30:35.606 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ssb 2023-07-14T11:30:35.606 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ssb/ssb.ko 2023-07-14T11:30:35.607 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target 2023-07-14T11:30:35.607 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/iscsi 2023-07-14T11:30:35.607 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-07-14T11:30:35.607 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/loopback 2023-07-14T11:30:35.607 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/loopback/tcm_loop.ko 2023-07-14T11:30:35.607 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_file.ko 2023-07-14T11:30:35.607 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_iblock.ko 2023-07-14T11:30:35.607 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_mod.ko 2023-07-14T11:30:35.608 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_pscsi.ko 2023-07-14T11:30:35.608 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_user.ko 2023-07-14T11:30:35.608 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/tcm_fc 2023-07-14T11:30:35.608 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-07-14T11:30:35.608 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal 2023-07-14T11:30:35.608 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel 2023-07-14T11:30:35.608 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal 2023-07-14T11:30:35.609 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-07-14T11:30:35.609 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-07-14T11:30:35.609 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-07-14T11:30:35.609 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-07-14T11:30:35.609 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-07-14T11:30:35.609 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-07-14T11:30:35.609 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-07-14T11:30:35.609 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-07-14T11:30:35.610 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-07-14T11:30:35.610 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-07-14T11:30:35.610 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-07-14T11:30:35.610 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-07-14T11:30:35.610 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-07-14T11:30:35.610 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-07-14T11:30:35.610 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-07-14T11:30:35.610 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty 2023-07-14T11:30:35.611 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/ipwireless 2023-07-14T11:30:35.611 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-07-14T11:30:35.611 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/n_gsm.ko 2023-07-14T11:30:35.611 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/n_hdlc.ko 2023-07-14T11:30:35.611 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/nozomi.ko 2023-07-14T11:30:35.611 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial 2023-07-14T11:30:35.611 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/8250 2023-07-14T11:30:35.611 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-07-14T11:30:35.612 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/arc_uart.ko 2023-07-14T11:30:35.612 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/jsm 2023-07-14T11:30:35.612 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/jsm/jsm.ko 2023-07-14T11:30:35.612 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/synclink_gt.ko 2023-07-14T11:30:35.612 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs 2023-07-14T11:30:35.612 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/core 2023-07-14T11:30:35.612 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/core/ufshcd-core.ko 2023-07-14T11:30:35.613 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/host 2023-07-14T11:30:35.613 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-07-14T11:30:35.613 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio 2023-07-14T11:30:35.613 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio.ko 2023-07-14T11:30:35.613 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_aec.ko 2023-07-14T11:30:35.613 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_cif.ko 2023-07-14T11:30:35.613 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_pci_generic.ko 2023-07-14T11:30:35.613 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_sercos3.ko 2023-07-14T11:30:35.614 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb 2023-07-14T11:30:35.614 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class 2023-07-14T11:30:35.614 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/cdc-acm.ko 2023-07-14T11:30:35.614 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/cdc-wdm.ko 2023-07-14T11:30:35.614 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/usblp.ko 2023-07-14T11:30:35.614 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/usbtmc.ko 2023-07-14T11:30:35.614 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/host 2023-07-14T11:30:35.614 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/host/sl811-hcd.ko 2023-07-14T11:30:35.615 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/image 2023-07-14T11:30:35.615 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/image/mdc800.ko 2023-07-14T11:30:35.615 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/image/microtek.ko 2023-07-14T11:30:35.615 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc 2023-07-14T11:30:35.615 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/adutux.ko 2023-07-14T11:30:35.615 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/appledisplay.ko 2023-07-14T11:30:35.615 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/emi26.ko 2023-07-14T11:30:35.615 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/emi62.ko 2023-07-14T11:30:35.616 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/ezusb.ko 2023-07-14T11:30:35.616 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/idmouse.ko 2023-07-14T11:30:35.616 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/iowarrior.ko 2023-07-14T11:30:35.616 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/isight_firmware.ko 2023-07-14T11:30:35.616 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/ldusb.ko 2023-07-14T11:30:35.616 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/legousbtower.ko 2023-07-14T11:30:35.616 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/sisusbvga 2023-07-14T11:30:35.616 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-07-14T11:30:35.617 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/trancevibrator.ko 2023-07-14T11:30:35.617 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/usb3503.ko 2023-07-14T11:30:35.617 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/usblcd.ko 2023-07-14T11:30:35.617 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/usbsevseg.ko 2023-07-14T11:30:35.617 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/uss720.ko 2023-07-14T11:30:35.617 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/yurex.ko 2023-07-14T11:30:35.617 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/phy 2023-07-14T11:30:35.617 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/phy/phy-generic.ko 2023-07-14T11:30:35.618 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial 2023-07-14T11:30:35.618 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/aircable.ko 2023-07-14T11:30:35.618 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ark3116.ko 2023-07-14T11:30:35.618 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/belkin_sa.ko 2023-07-14T11:30:35.618 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ch341.ko 2023-07-14T11:30:35.618 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/cp210x.ko 2023-07-14T11:30:35.618 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/cyberjack.ko 2023-07-14T11:30:35.618 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/cypress_m8.ko 2023-07-14T11:30:35.619 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/digi_acceleport.ko 2023-07-14T11:30:35.619 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/empeg.ko 2023-07-14T11:30:35.619 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ftdi_sio.ko 2023-07-14T11:30:35.619 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/garmin_gps.ko 2023-07-14T11:30:35.619 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/io_edgeport.ko 2023-07-14T11:30:35.619 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/io_ti.ko 2023-07-14T11:30:35.619 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ipaq.ko 2023-07-14T11:30:35.619 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ipw.ko 2023-07-14T11:30:35.620 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ir-usb.ko 2023-07-14T11:30:35.620 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-07-14T11:30:35.620 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/keyspan.ko 2023-07-14T11:30:35.620 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/keyspan_pda.ko 2023-07-14T11:30:35.620 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/kl5kusb105.ko 2023-07-14T11:30:35.620 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/kobil_sct.ko 2023-07-14T11:30:35.620 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/mct_u232.ko 2023-07-14T11:30:35.620 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/mos7720.ko 2023-07-14T11:30:35.621 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/mos7840.ko 2023-07-14T11:30:35.621 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/navman.ko 2023-07-14T11:30:35.621 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/omninet.ko 2023-07-14T11:30:35.621 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/opticon.ko 2023-07-14T11:30:35.621 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/option.ko 2023-07-14T11:30:35.621 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/oti6858.ko 2023-07-14T11:30:35.621 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/pl2303.ko 2023-07-14T11:30:35.621 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/qcaux.ko 2023-07-14T11:30:35.622 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/qcserial.ko 2023-07-14T11:30:35.622 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/quatech2.ko 2023-07-14T11:30:35.622 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/safe_serial.ko 2023-07-14T11:30:35.622 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/sierra.ko 2023-07-14T11:30:35.622 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/spcp8x5.ko 2023-07-14T11:30:35.622 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ssu100.ko 2023-07-14T11:30:35.622 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/symbolserial.ko 2023-07-14T11:30:35.622 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-07-14T11:30:35.623 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-07-14T11:30:35.623 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/usb_debug.ko 2023-07-14T11:30:35.623 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/usb_wwan.ko 2023-07-14T11:30:35.623 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/visor.ko 2023-07-14T11:30:35.623 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/whiteheat.ko 2023-07-14T11:30:35.623 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/xsens_mt.ko 2023-07-14T11:30:35.623 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage 2023-07-14T11:30:35.623 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/uas.ko 2023-07-14T11:30:35.624 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-alauda.ko 2023-07-14T11:30:35.624 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-cypress.ko 2023-07-14T11:30:35.624 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-datafab.ko 2023-07-14T11:30:35.624 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-07-14T11:30:35.624 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-freecom.ko 2023-07-14T11:30:35.624 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-isd200.ko 2023-07-14T11:30:35.624 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-07-14T11:30:35.625 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-karma.ko 2023-07-14T11:30:35.625 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-onetouch.ko 2023-07-14T11:30:35.625 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-realtek.ko 2023-07-14T11:30:35.625 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-sddr09.ko 2023-07-14T11:30:35.625 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-sddr55.ko 2023-07-14T11:30:35.625 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-usbat.ko 2023-07-14T11:30:35.625 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/usb-storage.ko 2023-07-14T11:30:35.625 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip 2023-07-14T11:30:35.626 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip/usbip-core.ko 2023-07-14T11:30:35.626 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip/usbip-host.ko 2023-07-14T11:30:35.626 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-07-14T11:30:35.626 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio 2023-07-14T11:30:35.626 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/pci 2023-07-14T11:30:35.626 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-07-14T11:30:35.626 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/pci/vfio-pci.ko 2023-07-14T11:30:35.626 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/vfio.ko 2023-07-14T11:30:35.627 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-07-14T11:30:35.627 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost 2023-07-14T11:30:35.627 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost.ko 2023-07-14T11:30:35.627 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost_iotlb.ko 2023-07-14T11:30:35.627 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost_net.ko 2023-07-14T11:30:35.627 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost_scsi.ko 2023-07-14T11:30:35.627 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video 2023-07-14T11:30:35.627 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight 2023-07-14T11:30:35.628 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/apple_bl.ko 2023-07-14T11:30:35.628 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/lcd.ko 2023-07-14T11:30:35.628 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/platform_lcd.ko 2023-07-14T11:30:35.628 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio 2023-07-14T11:30:35.628 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio.ko 2023-07-14T11:30:35.628 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_balloon.ko 2023-07-14T11:30:35.628 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_mmio.ko 2023-07-14T11:30:35.629 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_pci.ko 2023-07-14T11:30:35.629 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-07-14T11:30:35.629 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-07-14T11:30:35.629 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_ring.ko 2023-07-14T11:30:35.629 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1 2023-07-14T11:30:35.629 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/masters 2023-07-14T11:30:35.629 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/masters/ds2482.ko 2023-07-14T11:30:35.629 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/masters/ds2490.ko 2023-07-14T11:30:35.630 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves 2023-07-14T11:30:35.630 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-07-14T11:30:35.630 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-07-14T11:30:35.630 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-07-14T11:30:35.630 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-07-14T11:30:35.630 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-07-14T11:30:35.630 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-07-14T11:30:35.630 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-07-14T11:30:35.631 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-07-14T11:30:35.631 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-07-14T11:30:35.631 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_smem.ko 2023-07-14T11:30:35.631 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_therm.ko 2023-07-14T11:30:35.631 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/wire.ko 2023-07-14T11:30:35.631 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog 2023-07-14T11:30:35.631 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/alim1535_wdt.ko 2023-07-14T11:30:35.631 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/alim7101_wdt.ko 2023-07-14T11:30:35.632 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/f71808e_wdt.ko 2023-07-14T11:30:35.632 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/hpwdt.ko 2023-07-14T11:30:35.632 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/i6300esb.ko 2023-07-14T11:30:35.632 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-07-14T11:30:35.632 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/iTCO_wdt.ko 2023-07-14T11:30:35.632 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/ib700wdt.ko 2023-07-14T11:30:35.632 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/ibmasr.ko 2023-07-14T11:30:35.633 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-07-14T11:30:35.633 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/it8712f_wdt.ko 2023-07-14T11:30:35.633 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/it87_wdt.ko 2023-07-14T11:30:35.633 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/machzwd.ko 2023-07-14T11:30:35.633 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/nv_tco.ko 2023-07-14T11:30:35.633 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/pcwd_pci.ko 2023-07-14T11:30:35.633 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/pcwd_usb.ko 2023-07-14T11:30:35.633 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-07-14T11:30:35.634 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sch311x_wdt.ko 2023-07-14T11:30:35.634 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/softdog.ko 2023-07-14T11:30:35.634 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sp5100_tco.ko 2023-07-14T11:30:35.634 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/via_wdt.ko 2023-07-14T11:30:35.634 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-07-14T11:30:35.634 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/w83877f_wdt.ko 2023-07-14T11:30:35.634 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/w83977f_wdt.ko 2023-07-14T11:30:35.634 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/wdt_pci.ko 2023-07-14T11:30:35.635 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/xen_wdt.ko 2023-07-14T11:30:35.635 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen 2023-07-14T11:30:35.635 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-acpi-processor.ko 2023-07-14T11:30:35.635 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-evtchn.ko 2023-07-14T11:30:35.635 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-gntalloc.ko 2023-07-14T11:30:35.635 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-gntdev.ko 2023-07-14T11:30:35.635 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-pciback 2023-07-14T11:30:35.635 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-07-14T11:30:35.636 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-privcmd.ko 2023-07-14T11:30:35.636 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xenfs 2023-07-14T11:30:35.636 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xenfs/xenfs.ko 2023-07-14T11:30:35.636 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs 2023-07-14T11:30:35.636 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/9p 2023-07-14T11:30:35.636 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/9p/9p.ko 2023-07-14T11:30:35.636 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/affs 2023-07-14T11:30:35.636 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/affs/affs.ko 2023-07-14T11:30:35.637 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/befs 2023-07-14T11:30:35.637 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/befs/befs.ko 2023-07-14T11:30:35.637 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/binfmt_misc.ko 2023-07-14T11:30:35.637 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/btrfs 2023-07-14T11:30:35.637 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/btrfs/btrfs.ko 2023-07-14T11:30:35.637 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cachefiles 2023-07-14T11:30:35.637 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cachefiles/cachefiles.ko 2023-07-14T11:30:35.637 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ceph 2023-07-14T11:30:35.638 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ceph/ceph.ko 2023-07-14T11:30:35.638 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/coda 2023-07-14T11:30:35.638 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/coda/coda.ko 2023-07-14T11:30:35.638 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cramfs 2023-07-14T11:30:35.638 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cramfs/cramfs.ko 2023-07-14T11:30:35.638 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/dlm 2023-07-14T11:30:35.638 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/dlm/dlm.ko 2023-07-14T11:30:35.638 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ecryptfs 2023-07-14T11:30:35.639 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ecryptfs/ecryptfs.ko 2023-07-14T11:30:35.639 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/f2fs 2023-07-14T11:30:35.639 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/f2fs/f2fs.ko 2023-07-14T11:30:35.639 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat 2023-07-14T11:30:35.639 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat/fat.ko 2023-07-14T11:30:35.639 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat/msdos.ko 2023-07-14T11:30:35.639 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat/vfat.ko 2023-07-14T11:30:35.639 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fscache 2023-07-14T11:30:35.640 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fscache/fscache.ko 2023-07-14T11:30:35.640 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fuse 2023-07-14T11:30:35.640 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fuse/cuse.ko 2023-07-14T11:30:35.640 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fuse/fuse.ko 2023-07-14T11:30:35.640 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/gfs2 2023-07-14T11:30:35.640 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/gfs2/gfs2.ko 2023-07-14T11:30:35.640 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfs 2023-07-14T11:30:35.641 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfs/hfs.ko 2023-07-14T11:30:35.641 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfsplus 2023-07-14T11:30:35.641 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfsplus/hfsplus.ko 2023-07-14T11:30:35.641 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/isofs 2023-07-14T11:30:35.641 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/isofs/isofs.ko 2023-07-14T11:30:35.641 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/jfs 2023-07-14T11:30:35.641 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/jfs/jfs.ko 2023-07-14T11:30:35.641 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/lockd 2023-07-14T11:30:35.642 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/lockd/lockd.ko 2023-07-14T11:30:35.642 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/minix 2023-07-14T11:30:35.642 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/minix/minix.ko 2023-07-14T11:30:35.642 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/netfs 2023-07-14T11:30:35.642 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/netfs/netfs.ko 2023-07-14T11:30:35.642 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs 2023-07-14T11:30:35.642 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/blocklayout 2023-07-14T11:30:35.642 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-07-14T11:30:35.643 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/filelayout 2023-07-14T11:30:35.643 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-07-14T11:30:35.643 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/flexfilelayout 2023-07-14T11:30:35.643 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-07-14T11:30:35.643 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/nfs.ko 2023-07-14T11:30:35.643 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/nfsv3.ko 2023-07-14T11:30:35.643 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/nfsv4.ko 2023-07-14T11:30:35.643 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs_common 2023-07-14T11:30:35.644 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs_common/grace.ko 2023-07-14T11:30:35.644 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs_common/nfs_acl.ko 2023-07-14T11:30:35.644 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfsd 2023-07-14T11:30:35.644 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfsd/nfsd.ko 2023-07-14T11:30:35.644 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nilfs2 2023-07-14T11:30:35.644 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nilfs2/nilfs2.ko 2023-07-14T11:30:35.644 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls 2023-07-14T11:30:35.644 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-celtic.ko 2023-07-14T11:30:35.645 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-centeuro.ko 2023-07-14T11:30:35.645 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-croatian.ko 2023-07-14T11:30:35.645 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-cyrillic.ko 2023-07-14T11:30:35.645 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-gaelic.ko 2023-07-14T11:30:35.645 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-greek.ko 2023-07-14T11:30:35.645 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-iceland.ko 2023-07-14T11:30:35.645 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-inuit.ko 2023-07-14T11:30:35.645 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-roman.ko 2023-07-14T11:30:35.646 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-romanian.ko 2023-07-14T11:30:35.646 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-turkish.ko 2023-07-14T11:30:35.646 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp1250.ko 2023-07-14T11:30:35.646 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp1251.ko 2023-07-14T11:30:35.646 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp1255.ko 2023-07-14T11:30:35.646 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp737.ko 2023-07-14T11:30:35.646 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp775.ko 2023-07-14T11:30:35.646 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp850.ko 2023-07-14T11:30:35.647 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp852.ko 2023-07-14T11:30:35.647 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp855.ko 2023-07-14T11:30:35.647 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp857.ko 2023-07-14T11:30:35.647 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp860.ko 2023-07-14T11:30:35.647 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp861.ko 2023-07-14T11:30:35.647 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp862.ko 2023-07-14T11:30:35.647 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp863.ko 2023-07-14T11:30:35.647 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp864.ko 2023-07-14T11:30:35.648 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp865.ko 2023-07-14T11:30:35.648 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp866.ko 2023-07-14T11:30:35.648 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp869.ko 2023-07-14T11:30:35.648 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp874.ko 2023-07-14T11:30:35.648 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp932.ko 2023-07-14T11:30:35.648 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp936.ko 2023-07-14T11:30:35.648 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp949.ko 2023-07-14T11:30:35.649 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp950.ko 2023-07-14T11:30:35.649 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_euc-jp.ko 2023-07-14T11:30:35.649 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-1.ko 2023-07-14T11:30:35.649 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-13.ko 2023-07-14T11:30:35.649 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-14.ko 2023-07-14T11:30:35.649 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-15.ko 2023-07-14T11:30:35.649 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-2.ko 2023-07-14T11:30:35.649 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-3.ko 2023-07-14T11:30:35.650 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-4.ko 2023-07-14T11:30:35.650 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-5.ko 2023-07-14T11:30:35.650 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-6.ko 2023-07-14T11:30:35.650 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-7.ko 2023-07-14T11:30:35.650 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-9.ko 2023-07-14T11:30:35.650 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_koi8-r.ko 2023-07-14T11:30:35.650 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_koi8-ru.ko 2023-07-14T11:30:35.650 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_koi8-u.ko 2023-07-14T11:30:35.651 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_utf8.ko 2023-07-14T11:30:35.651 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2 2023-07-14T11:30:35.651 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/cluster 2023-07-14T11:30:35.651 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-07-14T11:30:35.651 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlm 2023-07-14T11:30:35.651 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-07-14T11:30:35.651 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlmfs 2023-07-14T11:30:35.651 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-07-14T11:30:35.652 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2.ko 2023-07-14T11:30:35.652 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-07-14T11:30:35.652 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-07-14T11:30:35.652 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-07-14T11:30:35.652 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/overlayfs 2023-07-14T11:30:35.652 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/overlayfs/overlay.ko 2023-07-14T11:30:35.652 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/pstore 2023-07-14T11:30:35.652 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/pstore/ramoops.ko 2023-07-14T11:30:35.653 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/reiserfs 2023-07-14T11:30:35.653 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/reiserfs/reiserfs.ko 2023-07-14T11:30:35.653 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/romfs 2023-07-14T11:30:35.653 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/romfs/romfs.ko 2023-07-14T11:30:35.653 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb 2023-07-14T11:30:35.653 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/client 2023-07-14T11:30:35.653 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/client/cifs.ko 2023-07-14T11:30:35.653 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/common 2023-07-14T11:30:35.654 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/common/cifs_arc4.ko 2023-07-14T11:30:35.654 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/common/cifs_md4.ko 2023-07-14T11:30:35.654 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/squashfs 2023-07-14T11:30:35.654 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/squashfs/squashfs.ko 2023-07-14T11:30:35.654 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/sysv 2023-07-14T11:30:35.654 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/sysv/sysv.ko 2023-07-14T11:30:35.654 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/udf 2023-07-14T11:30:35.654 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/udf/udf.ko 2023-07-14T11:30:35.655 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ufs 2023-07-14T11:30:35.655 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ufs/ufs.ko 2023-07-14T11:30:35.655 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/xfs 2023-07-14T11:30:35.655 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/xfs/xfs.ko 2023-07-14T11:30:35.655 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel 2023-07-14T11:30:35.655 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/rcu 2023-07-14T11:30:35.655 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/rcu/rcutorture.ko 2023-07-14T11:30:35.655 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/torture.ko 2023-07-14T11:30:35.656 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/trace 2023-07-14T11:30:35.656 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-07-14T11:30:35.656 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib 2023-07-14T11:30:35.656 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/asn1_encoder.ko 2023-07-14T11:30:35.656 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crc-itu-t.ko 2023-07-14T11:30:35.656 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crc8.ko 2023-07-14T11:30:35.656 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crypto 2023-07-14T11:30:35.657 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crypto/libarc4.ko 2023-07-14T11:30:35.657 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crypto/libdes.ko 2023-07-14T11:30:35.657 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/libcrc32c.ko 2023-07-14T11:30:35.657 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lru_cache.ko 2023-07-14T11:30:35.657 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lz4 2023-07-14T11:30:35.657 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lz4/lz4_compress.ko 2023-07-14T11:30:35.657 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lz4/lz4hc_compress.ko 2023-07-14T11:30:35.657 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/math 2023-07-14T11:30:35.658 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/math/cordic.ko 2023-07-14T11:30:35.658 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/raid6 2023-07-14T11:30:35.658 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/raid6/raid6_pq.ko 2023-07-14T11:30:35.658 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/reed_solomon 2023-07-14T11:30:35.658 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/reed_solomon/reed_solomon.ko 2023-07-14T11:30:35.658 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/ts_bm.ko 2023-07-14T11:30:35.659 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/ts_fsm.ko 2023-07-14T11:30:35.659 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/ts_kmp.ko 2023-07-14T11:30:35.659 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/zstd 2023-07-14T11:30:35.659 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/zstd/zstd_compress.ko 2023-07-14T11:30:35.659 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/mm 2023-07-14T11:30:35.659 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/mm/hwpoison-inject.ko 2023-07-14T11:30:35.659 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net 2023-07-14T11:30:35.659 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan 2023-07-14T11:30:35.660 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/6lowpan.ko 2023-07-14T11:30:35.660 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_dest.ko 2023-07-14T11:30:35.660 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_fragment.ko 2023-07-14T11:30:35.660 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_hop.ko 2023-07-14T11:30:35.660 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_ipv6.ko 2023-07-14T11:30:35.660 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_mobility.ko 2023-07-14T11:30:35.660 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_routing.ko 2023-07-14T11:30:35.660 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_udp.ko 2023-07-14T11:30:35.661 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802 2023-07-14T11:30:35.661 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/garp.ko 2023-07-14T11:30:35.661 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/mrp.ko 2023-07-14T11:30:35.661 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/p8022.ko 2023-07-14T11:30:35.661 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/psnap.ko 2023-07-14T11:30:35.661 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/stp.ko 2023-07-14T11:30:35.661 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/8021q 2023-07-14T11:30:35.661 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/8021q/8021q.ko 2023-07-14T11:30:35.662 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p 2023-07-14T11:30:35.662 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet.ko 2023-07-14T11:30:35.662 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet_fd.ko 2023-07-14T11:30:35.662 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet_rdma.ko 2023-07-14T11:30:35.662 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet_virtio.ko 2023-07-14T11:30:35.662 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/batman-adv 2023-07-14T11:30:35.662 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/batman-adv/batman-adv.ko 2023-07-14T11:30:35.662 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge 2023-07-14T11:30:35.663 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/br_netfilter.ko 2023-07-14T11:30:35.663 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/bridge.ko 2023-07-14T11:30:35.663 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter 2023-07-14T11:30:35.663 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-07-14T11:30:35.663 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_among.ko 2023-07-14T11:30:35.663 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_arp.ko 2023-07-14T11:30:35.663 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-07-14T11:30:35.663 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-07-14T11:30:35.664 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_ip.ko 2023-07-14T11:30:35.664 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-07-14T11:30:35.664 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_limit.ko 2023-07-14T11:30:35.664 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_log.ko 2023-07-14T11:30:35.664 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_mark.ko 2023-07-14T11:30:35.664 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-07-14T11:30:35.664 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-07-14T11:30:35.665 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-07-14T11:30:35.665 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-07-14T11:30:35.665 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_snat.ko 2023-07-14T11:30:35.665 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_stp.ko 2023-07-14T11:30:35.665 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-07-14T11:30:35.665 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-07-14T11:30:35.665 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-07-14T11:30:35.665 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-07-14T11:30:35.666 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtables.ko 2023-07-14T11:30:35.666 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-07-14T11:30:35.666 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-07-14T11:30:35.666 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-07-14T11:30:35.666 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ceph 2023-07-14T11:30:35.666 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ceph/libceph.ko 2023-07-14T11:30:35.666 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/core 2023-07-14T11:30:35.666 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/core/failover.ko 2023-07-14T11:30:35.667 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/core/pktgen.ko 2023-07-14T11:30:35.667 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp 2023-07-14T11:30:35.667 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp.ko 2023-07-14T11:30:35.667 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp_diag.ko 2023-07-14T11:30:35.667 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp_ipv4.ko 2023-07-14T11:30:35.667 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp_ipv6.ko 2023-07-14T11:30:35.667 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dns_resolver 2023-07-14T11:30:35.667 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dns_resolver/dns_resolver.ko 2023-07-14T11:30:35.668 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa 2023-07-14T11:30:35.668 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/dsa_core.ko 2023-07-14T11:30:35.668 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_brcm.ko 2023-07-14T11:30:35.668 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_dsa.ko 2023-07-14T11:30:35.668 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_none.ko 2023-07-14T11:30:35.668 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_trailer.ko 2023-07-14T11:30:35.668 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154 2023-07-14T11:30:35.668 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/6lowpan 2023-07-14T11:30:35.669 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-07-14T11:30:35.669 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/ieee802154.ko 2023-07-14T11:30:35.669 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/ieee802154_socket.ko 2023-07-14T11:30:35.669 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4 2023-07-14T11:30:35.669 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ah4.ko 2023-07-14T11:30:35.669 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/esp4.ko 2023-07-14T11:30:35.669 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/fou.ko 2023-07-14T11:30:35.669 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/gre.ko 2023-07-14T11:30:35.670 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/inet_diag.ko 2023-07-14T11:30:35.670 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ip_gre.ko 2023-07-14T11:30:35.670 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ip_tunnel.ko 2023-07-14T11:30:35.670 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ip_vti.ko 2023-07-14T11:30:35.670 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ipcomp.ko 2023-07-14T11:30:35.670 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ipip.ko 2023-07-14T11:30:35.670 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter 2023-07-14T11:30:35.670 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/arp_tables.ko 2023-07-14T11:30:35.671 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-07-14T11:30:35.671 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-07-14T11:30:35.671 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-07-14T11:30:35.671 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-07-14T11:30:35.671 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-07-14T11:30:35.671 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-07-14T11:30:35.671 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-07-14T11:30:35.671 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-07-14T11:30:35.672 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-07-14T11:30:35.672 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_security.ko 2023-07-14T11:30:35.672 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-07-14T11:30:35.672 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-07-14T11:30:35.672 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-07-14T11:30:35.672 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-07-14T11:30:35.672 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-07-14T11:30:35.673 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-07-14T11:30:35.673 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-07-14T11:30:35.673 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-07-14T11:30:35.673 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-07-14T11:30:35.673 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-07-14T11:30:35.673 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_bic.ko 2023-07-14T11:30:35.673 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_dctcp.ko 2023-07-14T11:30:35.673 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_diag.ko 2023-07-14T11:30:35.674 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_highspeed.ko 2023-07-14T11:30:35.674 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_htcp.ko 2023-07-14T11:30:35.674 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_hybla.ko 2023-07-14T11:30:35.674 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_illinois.ko 2023-07-14T11:30:35.674 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_lp.ko 2023-07-14T11:30:35.674 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_scalable.ko 2023-07-14T11:30:35.674 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_vegas.ko 2023-07-14T11:30:35.674 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_veno.ko 2023-07-14T11:30:35.675 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_westwood.ko 2023-07-14T11:30:35.675 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_yeah.ko 2023-07-14T11:30:35.675 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tunnel4.ko 2023-07-14T11:30:35.675 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/udp_diag.ko 2023-07-14T11:30:35.675 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/udp_tunnel.ko 2023-07-14T11:30:35.675 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/xfrm4_tunnel.ko 2023-07-14T11:30:35.675 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6 2023-07-14T11:30:35.675 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ah6.ko 2023-07-14T11:30:35.676 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/esp6.ko 2023-07-14T11:30:35.676 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/fou6.ko 2023-07-14T11:30:35.676 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ip6_tunnel.ko 2023-07-14T11:30:35.676 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-07-14T11:30:35.676 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ip6_vti.ko 2023-07-14T11:30:35.676 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ipcomp6.ko 2023-07-14T11:30:35.676 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter 2023-07-14T11:30:35.676 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-07-14T11:30:35.677 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-07-14T11:30:35.677 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-07-14T11:30:35.677 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-07-14T11:30:35.677 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-07-14T11:30:35.677 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-07-14T11:30:35.677 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-07-14T11:30:35.677 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-07-14T11:30:35.677 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-07-14T11:30:35.678 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-07-14T11:30:35.678 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-07-14T11:30:35.678 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-07-14T11:30:35.678 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-07-14T11:30:35.678 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-07-14T11:30:35.678 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-07-14T11:30:35.678 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-07-14T11:30:35.678 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-07-14T11:30:35.679 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-07-14T11:30:35.679 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-07-14T11:30:35.679 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-07-14T11:30:35.679 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-07-14T11:30:35.679 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-07-14T11:30:35.679 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-07-14T11:30:35.679 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-07-14T11:30:35.679 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/sit.ko 2023-07-14T11:30:35.680 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/tunnel6.ko 2023-07-14T11:30:35.680 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/xfrm6_tunnel.ko 2023-07-14T11:30:35.680 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/key 2023-07-14T11:30:35.680 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/key/af_key.ko 2023-07-14T11:30:35.680 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp 2023-07-14T11:30:35.680 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_core.ko 2023-07-14T11:30:35.680 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_debugfs.ko 2023-07-14T11:30:35.680 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_eth.ko 2023-07-14T11:30:35.681 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_ip.ko 2023-07-14T11:30:35.681 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_ip6.ko 2023-07-14T11:30:35.681 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_netlink.ko 2023-07-14T11:30:35.681 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_ppp.ko 2023-07-14T11:30:35.681 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/llc 2023-07-14T11:30:35.681 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/llc/llc.ko 2023-07-14T11:30:35.681 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mac802154 2023-07-14T11:30:35.681 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mac802154/mac802154.ko 2023-07-14T11:30:35.682 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mpls 2023-07-14T11:30:35.682 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mpls/mpls_gso.ko 2023-07-14T11:30:35.682 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter 2023-07-14T11:30:35.682 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset 2023-07-14T11:30:35.682 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set.ko 2023-07-14T11:30:35.682 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-07-14T11:30:35.682 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-07-14T11:30:35.682 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-07-14T11:30:35.683 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-07-14T11:30:35.683 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-07-14T11:30:35.683 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-07-14T11:30:35.683 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-07-14T11:30:35.683 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-07-14T11:30:35.683 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-07-14T11:30:35.683 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-07-14T11:30:35.683 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-07-14T11:30:35.684 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-07-14T11:30:35.684 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-07-14T11:30:35.684 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-07-14T11:30:35.684 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-07-14T11:30:35.684 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs 2023-07-14T11:30:35.684 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs.ko 2023-07-14T11:30:35.684 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-07-14T11:30:35.685 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-07-14T11:30:35.685 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-07-14T11:30:35.685 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-07-14T11:30:35.685 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-07-14T11:30:35.685 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-07-14T11:30:35.685 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-07-14T11:30:35.685 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-07-14T11:30:35.685 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-07-14T11:30:35.686 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-07-14T11:30:35.686 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-07-14T11:30:35.686 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-07-14T11:30:35.686 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-07-14T11:30:35.686 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conncount.ko 2023-07-14T11:30:35.686 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack.ko 2023-07-14T11:30:35.686 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-07-14T11:30:35.686 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-07-14T11:30:35.687 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-07-14T11:30:35.687 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_h323.ko 2023-07-14T11:30:35.687 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_irc.ko 2023-07-14T11:30:35.687 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-07-14T11:30:35.687 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-07-14T11:30:35.687 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-07-14T11:30:35.687 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_sane.ko 2023-07-14T11:30:35.687 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_sip.ko 2023-07-14T11:30:35.688 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-07-14T11:30:35.688 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-07-14T11:30:35.688 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_dup_netdev.ko 2023-07-14T11:30:35.688 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_flow_table.ko 2023-07-14T11:30:35.688 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_flow_table_inet.ko 2023-07-14T11:30:35.688 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_log_syslog.ko 2023-07-14T11:30:35.688 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat.ko 2023-07-14T11:30:35.688 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_amanda.ko 2023-07-14T11:30:35.689 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_ftp.ko 2023-07-14T11:30:35.689 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_irc.ko 2023-07-14T11:30:35.689 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_sip.ko 2023-07-14T11:30:35.689 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_tftp.ko 2023-07-14T11:30:35.689 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_synproxy_core.ko 2023-07-14T11:30:35.689 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_tables.ko 2023-07-14T11:30:35.689 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink.ko 2023-07-14T11:30:35.689 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_acct.ko 2023-07-14T11:30:35.690 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_log.ko 2023-07-14T11:30:35.690 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_osf.ko 2023-07-14T11:30:35.690 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_queue.ko 2023-07-14T11:30:35.690 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_chain_nat.ko 2023-07-14T11:30:35.690 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_compat.ko 2023-07-14T11:30:35.690 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_ct.ko 2023-07-14T11:30:35.690 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_dup_netdev.ko 2023-07-14T11:30:35.690 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_fib.ko 2023-07-14T11:30:35.691 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_fwd_netdev.ko 2023-07-14T11:30:35.691 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_hash.ko 2023-07-14T11:30:35.691 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_limit.ko 2023-07-14T11:30:35.691 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_log.ko 2023-07-14T11:30:35.691 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_masq.ko 2023-07-14T11:30:35.691 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_nat.ko 2023-07-14T11:30:35.691 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_numgen.ko 2023-07-14T11:30:35.691 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_queue.ko 2023-07-14T11:30:35.692 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_quota.ko 2023-07-14T11:30:35.692 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_redir.ko 2023-07-14T11:30:35.692 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_reject.ko 2023-07-14T11:30:35.692 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_reject_inet.ko 2023-07-14T11:30:35.692 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_socket.ko 2023-07-14T11:30:35.692 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_tunnel.ko 2023-07-14T11:30:35.692 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_AUDIT.ko 2023-07-14T11:30:35.693 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CHECKSUM.ko 2023-07-14T11:30:35.693 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CLASSIFY.ko 2023-07-14T11:30:35.693 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-07-14T11:30:35.693 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CT.ko 2023-07-14T11:30:35.693 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_DSCP.ko 2023-07-14T11:30:35.693 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_HL.ko 2023-07-14T11:30:35.693 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_HMARK.ko 2023-07-14T11:30:35.693 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_IDLETIMER.ko 2023-07-14T11:30:35.694 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_LED.ko 2023-07-14T11:30:35.694 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_LOG.ko 2023-07-14T11:30:35.694 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_MASQUERADE.ko 2023-07-14T11:30:35.694 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_NETMAP.ko 2023-07-14T11:30:35.694 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_NFLOG.ko 2023-07-14T11:30:35.694 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_NFQUEUE.ko 2023-07-14T11:30:35.694 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_RATEEST.ko 2023-07-14T11:30:35.694 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_REDIRECT.ko 2023-07-14T11:30:35.695 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_SECMARK.ko 2023-07-14T11:30:35.695 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TCPMSS.ko 2023-07-14T11:30:35.695 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-07-14T11:30:35.695 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TEE.ko 2023-07-14T11:30:35.695 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TPROXY.ko 2023-07-14T11:30:35.695 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TRACE.ko 2023-07-14T11:30:35.695 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_addrtype.ko 2023-07-14T11:30:35.695 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_bpf.ko 2023-07-14T11:30:35.696 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_cgroup.ko 2023-07-14T11:30:35.696 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_cluster.ko 2023-07-14T11:30:35.696 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_comment.ko 2023-07-14T11:30:35.696 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connbytes.ko 2023-07-14T11:30:35.696 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connlabel.ko 2023-07-14T11:30:35.696 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connlimit.ko 2023-07-14T11:30:35.696 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connmark.ko 2023-07-14T11:30:35.696 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_conntrack.ko 2023-07-14T11:30:35.697 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_cpu.ko 2023-07-14T11:30:35.697 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_dccp.ko 2023-07-14T11:30:35.697 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_devgroup.ko 2023-07-14T11:30:35.697 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_dscp.ko 2023-07-14T11:30:35.697 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_ecn.ko 2023-07-14T11:30:35.697 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_esp.ko 2023-07-14T11:30:35.697 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_hashlimit.ko 2023-07-14T11:30:35.697 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_helper.ko 2023-07-14T11:30:35.698 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_hl.ko 2023-07-14T11:30:35.698 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_ipcomp.ko 2023-07-14T11:30:35.698 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_iprange.ko 2023-07-14T11:30:35.698 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_ipvs.ko 2023-07-14T11:30:35.698 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_l2tp.ko 2023-07-14T11:30:35.698 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_length.ko 2023-07-14T11:30:35.698 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_limit.ko 2023-07-14T11:30:35.698 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_mac.ko 2023-07-14T11:30:35.699 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_mark.ko 2023-07-14T11:30:35.699 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_multiport.ko 2023-07-14T11:30:35.699 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_nat.ko 2023-07-14T11:30:35.699 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_nfacct.ko 2023-07-14T11:30:35.699 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_osf.ko 2023-07-14T11:30:35.699 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_owner.ko 2023-07-14T11:30:35.699 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_physdev.ko 2023-07-14T11:30:35.699 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_pkttype.ko 2023-07-14T11:30:35.700 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_policy.ko 2023-07-14T11:30:35.700 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_quota.ko 2023-07-14T11:30:35.700 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_rateest.ko 2023-07-14T11:30:35.700 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_realm.ko 2023-07-14T11:30:35.700 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_recent.ko 2023-07-14T11:30:35.700 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_sctp.ko 2023-07-14T11:30:35.700 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_set.ko 2023-07-14T11:30:35.701 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_socket.ko 2023-07-14T11:30:35.701 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_state.ko 2023-07-14T11:30:35.701 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_statistic.ko 2023-07-14T11:30:35.701 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_string.ko 2023-07-14T11:30:35.701 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_tcpmss.ko 2023-07-14T11:30:35.701 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_time.ko 2023-07-14T11:30:35.701 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_u32.ko 2023-07-14T11:30:35.701 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netlink 2023-07-14T11:30:35.702 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netlink/netlink_diag.ko 2023-07-14T11:30:35.702 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/nsh 2023-07-14T11:30:35.702 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/nsh/nsh.ko 2023-07-14T11:30:35.702 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch 2023-07-14T11:30:35.702 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/openvswitch.ko 2023-07-14T11:30:35.702 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/vport-geneve.ko 2023-07-14T11:30:35.702 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/vport-gre.ko 2023-07-14T11:30:35.702 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/vport-vxlan.ko 2023-07-14T11:30:35.703 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/packet 2023-07-14T11:30:35.703 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/packet/af_packet_diag.ko 2023-07-14T11:30:35.703 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds 2023-07-14T11:30:35.703 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds/rds.ko 2023-07-14T11:30:35.703 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds/rds_rdma.ko 2023-07-14T11:30:35.703 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds/rds_tcp.ko 2023-07-14T11:30:35.703 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched 2023-07-14T11:30:35.703 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_csum.ko 2023-07-14T11:30:35.704 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_gact.ko 2023-07-14T11:30:35.704 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_ipt.ko 2023-07-14T11:30:35.704 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_mirred.ko 2023-07-14T11:30:35.704 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_nat.ko 2023-07-14T11:30:35.704 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_pedit.ko 2023-07-14T11:30:35.704 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_police.ko 2023-07-14T11:30:35.704 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_simple.ko 2023-07-14T11:30:35.704 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_skbedit.ko 2023-07-14T11:30:35.705 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_vlan.ko 2023-07-14T11:30:35.705 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_basic.ko 2023-07-14T11:30:35.705 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_bpf.ko 2023-07-14T11:30:35.705 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_flow.ko 2023-07-14T11:30:35.705 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_fw.ko 2023-07-14T11:30:35.705 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_route.ko 2023-07-14T11:30:35.705 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_u32.ko 2023-07-14T11:30:35.705 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_cmp.ko 2023-07-14T11:30:35.706 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_ipset.ko 2023-07-14T11:30:35.706 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_meta.ko 2023-07-14T11:30:35.706 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_nbyte.ko 2023-07-14T11:30:35.706 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_text.ko 2023-07-14T11:30:35.706 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_u32.ko 2023-07-14T11:30:35.706 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_choke.ko 2023-07-14T11:30:35.706 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_codel.ko 2023-07-14T11:30:35.706 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_drr.ko 2023-07-14T11:30:35.707 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_fq.ko 2023-07-14T11:30:35.707 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_gred.ko 2023-07-14T11:30:35.707 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_hfsc.ko 2023-07-14T11:30:35.707 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_hhf.ko 2023-07-14T11:30:35.707 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_htb.ko 2023-07-14T11:30:35.707 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_ingress.ko 2023-07-14T11:30:35.707 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_mqprio.ko 2023-07-14T11:30:35.707 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_mqprio_lib.ko 2023-07-14T11:30:35.708 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_multiq.ko 2023-07-14T11:30:35.708 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_netem.ko 2023-07-14T11:30:35.708 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_pie.ko 2023-07-14T11:30:35.708 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_plug.ko 2023-07-14T11:30:35.708 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_prio.ko 2023-07-14T11:30:35.708 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_qfq.ko 2023-07-14T11:30:35.708 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_red.ko 2023-07-14T11:30:35.708 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_sfb.ko 2023-07-14T11:30:35.709 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_sfq.ko 2023-07-14T11:30:35.709 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_tbf.ko 2023-07-14T11:30:35.709 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_teql.ko 2023-07-14T11:30:35.709 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sctp 2023-07-14T11:30:35.709 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sctp/sctp.ko 2023-07-14T11:30:35.709 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sctp/sctp_diag.ko 2023-07-14T11:30:35.709 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc 2023-07-14T11:30:35.709 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/auth_gss 2023-07-14T11:30:35.710 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-07-14T11:30:35.710 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-07-14T11:30:35.710 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/sunrpc.ko 2023-07-14T11:30:35.710 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/xprtrdma 2023-07-14T11:30:35.710 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-07-14T11:30:35.710 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/tipc 2023-07-14T11:30:35.710 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/tipc/diag.ko 2023-07-14T11:30:35.710 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/tipc/tipc.ko 2023-07-14T11:30:35.711 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/unix 2023-07-14T11:30:35.711 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/unix/unix_diag.ko 2023-07-14T11:30:35.711 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock 2023-07-14T11:30:35.711 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-07-14T11:30:35.711 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-07-14T11:30:35.711 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vsock.ko 2023-07-14T11:30:35.711 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vsock_diag.ko 2023-07-14T11:30:35.711 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vsock_loopback.ko 2023-07-14T11:30:35.712 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/xfrm 2023-07-14T11:30:35.712 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/xfrm/xfrm_ipcomp.ko 2023-07-14T11:30:35.712 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security 2023-07-14T11:30:35.712 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys 2023-07-14T11:30:35.712 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/encrypted-keys 2023-07-14T11:30:35.712 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-07-14T11:30:35.712 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/trusted-keys 2023-07-14T11:30:35.713 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/trusted-keys/trusted.ko 2023-07-14T11:30:35.713 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/virt 2023-07-14T11:30:35.713 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/virt/lib 2023-07-14T11:30:35.713 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/virt/lib/irqbypass.ko 2023-07-14T11:30:35.713 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.alias 2023-07-14T11:30:35.713 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.alias.bin 2023-07-14T11:30:35.713 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.builtin 2023-07-14T11:30:35.713 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.builtin.bin 2023-07-14T11:30:35.714 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.builtin.modinfo 2023-07-14T11:30:35.714 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.dep 2023-07-14T11:30:35.714 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.dep.bin 2023-07-14T11:30:35.714 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.devname 2023-07-14T11:30:35.714 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.order 2023-07-14T11:30:35.714 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.softdep 2023-07-14T11:30:35.714 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.symbols 2023-07-14T11:30:35.714 INFO:teuthology.orchestra.run.smithi017.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.symbols.bin 2023-07-14T11:30:35.715 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc1-ga440c79eee02 2023-07-14T11:30:35.715 DEBUG:teuthology.orchestra.run.smithi017:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-07-14T11:30:35.743 INFO:teuthology.orchestra.run.smithi017.stdout:postinstall scriptlet (using /bin/sh): 2023-07-14T11:30:35.744 INFO:teuthology.orchestra.run.smithi017.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc1-ga440c79eee02 -a -r /boot/System.map-6.5.0-rc1-ga440c79eee02 ]; then 2023-07-14T11:30:35.744 INFO:teuthology.orchestra.run.smithi017.stdout:cp /boot/vmlinuz-6.5.0-rc1-ga440c79eee02 /boot/.vmlinuz-6.5.0-rc1-ga440c79eee02-rpm 2023-07-14T11:30:35.744 INFO:teuthology.orchestra.run.smithi017.stdout:cp /boot/System.map-6.5.0-rc1-ga440c79eee02 /boot/.System.map-6.5.0-rc1-ga440c79eee02-rpm 2023-07-14T11:30:35.744 INFO:teuthology.orchestra.run.smithi017.stdout:rm -f /boot/vmlinuz-6.5.0-rc1-ga440c79eee02 /boot/System.map-6.5.0-rc1-ga440c79eee02 2023-07-14T11:30:35.744 INFO:teuthology.orchestra.run.smithi017.stdout:/sbin/installkernel 6.5.0-rc1-ga440c79eee02 /boot/.vmlinuz-6.5.0-rc1-ga440c79eee02-rpm /boot/.System.map-6.5.0-rc1-ga440c79eee02-rpm 2023-07-14T11:30:35.745 INFO:teuthology.orchestra.run.smithi017.stdout:rm -f /boot/.vmlinuz-6.5.0-rc1-ga440c79eee02-rpm /boot/.System.map-6.5.0-rc1-ga440c79eee02-rpm 2023-07-14T11:30:35.745 INFO:teuthology.orchestra.run.smithi017.stdout:fi 2023-07-14T11:30:35.745 INFO:teuthology.orchestra.run.smithi017.stdout:preuninstall scriptlet (using /bin/sh): 2023-07-14T11:30:35.745 INFO:teuthology.orchestra.run.smithi017.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-07-14T11:30:35.745 INFO:teuthology.orchestra.run.smithi017.stdout:new-kernel-pkg --remove 6.5.0-rc1-ga440c79eee02 --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc1-ga440c79eee02.img 2023-07-14T11:30:35.745 INFO:teuthology.orchestra.run.smithi017.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-07-14T11:30:35.745 INFO:teuthology.orchestra.run.smithi017.stdout:kernel-install remove 6.5.0-rc1-ga440c79eee02 2023-07-14T11:30:35.745 INFO:teuthology.orchestra.run.smithi017.stdout:fi 2023-07-14T11:30:35.746 INFO:teuthology.orchestra.run.smithi017.stdout:postuninstall scriptlet (using /bin/sh): 2023-07-14T11:30:35.746 INFO:teuthology.orchestra.run.smithi017.stdout:if [ -x /sbin/update-bootloader ]; then 2023-07-14T11:30:35.746 INFO:teuthology.orchestra.run.smithi017.stdout:/sbin/update-bootloader --remove 6.5.0-rc1-ga440c79eee02 2023-07-14T11:30:35.746 INFO:teuthology.orchestra.run.smithi017.stdout:fi 2023-07-14T11:30:35.747 DEBUG:teuthology.orchestra.run.smithi017:> sudo rpm -qi grub2-tools 2023-07-14T11:30:35.837 INFO:teuthology.orchestra.run.smithi017.stdout:Name : grub2-tools 2023-07-14T11:30:35.837 INFO:teuthology.orchestra.run.smithi017.stdout:Epoch : 1 2023-07-14T11:30:35.837 INFO:teuthology.orchestra.run.smithi017.stdout:Version : 2.02 2023-07-14T11:30:35.838 INFO:teuthology.orchestra.run.smithi017.stdout:Release : 123.el8 2023-07-14T11:30:35.838 INFO:teuthology.orchestra.run.smithi017.stdout:Architecture: x86_64 2023-07-14T11:30:35.838 INFO:teuthology.orchestra.run.smithi017.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-07-14T11:30:35.838 INFO:teuthology.orchestra.run.smithi017.stdout:Group : System Environment/Base 2023-07-14T11:30:35.838 INFO:teuthology.orchestra.run.smithi017.stdout:Size : 9265450 2023-07-14T11:30:35.838 INFO:teuthology.orchestra.run.smithi017.stdout:License : GPLv3+ 2023-07-14T11:30:35.838 INFO:teuthology.orchestra.run.smithi017.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-07-14T11:30:35.838 INFO:teuthology.orchestra.run.smithi017.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-07-14T11:30:35.839 INFO:teuthology.orchestra.run.smithi017.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-07-14T11:30:35.839 INFO:teuthology.orchestra.run.smithi017.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-07-14T11:30:35.839 INFO:teuthology.orchestra.run.smithi017.stdout:Relocations : (not relocatable) 2023-07-14T11:30:35.839 INFO:teuthology.orchestra.run.smithi017.stdout:Packager : Red Hat, Inc. 2023-07-14T11:30:35.839 INFO:teuthology.orchestra.run.smithi017.stdout:Vendor : Red Hat, Inc. 2023-07-14T11:30:35.839 INFO:teuthology.orchestra.run.smithi017.stdout:URL : http://www.gnu.org/software/grub/ 2023-07-14T11:30:35.839 INFO:teuthology.orchestra.run.smithi017.stdout:Summary : Support tools for GRUB. 2023-07-14T11:30:35.840 INFO:teuthology.orchestra.run.smithi017.stdout:Description : 2023-07-14T11:30:35.840 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:30:35.840 INFO:teuthology.orchestra.run.smithi017.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-07-14T11:30:35.840 INFO:teuthology.orchestra.run.smithi017.stdout:customizable bootloader with modular architecture. It supports a rich 2023-07-14T11:30:35.840 INFO:teuthology.orchestra.run.smithi017.stdout:variety of kernel formats, file systems, computer architectures and 2023-07-14T11:30:35.840 INFO:teuthology.orchestra.run.smithi017.stdout:hardware devices. 2023-07-14T11:30:35.840 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:30:35.840 INFO:teuthology.orchestra.run.smithi017.stdout:This subpackage provides tools for support of all platforms. 2023-07-14T11:30:35.841 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-07-14T11:30:35.842 INFO:teuthology.task.kernel:Updating grub on smithi017 to boot 6.5.0-rc1-ga440c79eee02 2023-07-14T11:30:35.842 DEBUG:teuthology.orchestra.run.smithi017:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-07-14T11:30:38.893 INFO:teuthology.orchestra.run.smithi017.stderr:Generating grub configuration file ... 2023-07-14T11:30:41.264 INFO:teuthology.orchestra.run.smithi017.stderr:done 2023-07-14T11:30:41.266 DEBUG:teuthology.orchestra.run.smithi017:> mktemp 2023-07-14T11:30:41.290 INFO:teuthology.orchestra.run.smithi017.stdout:/tmp/tmp.RNNru7US7m 2023-07-14T11:30:41.290 DEBUG:teuthology.orchestra.run.smithi017:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.RNNru7US7m 2023-07-14T11:30:41.362 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 0666 /tmp/tmp.RNNru7US7m 2023-07-14T11:30:41.504 DEBUG:teuthology.orchestra.remote:smithi017:/tmp/tmp.RNNru7US7m is 6KB 2023-07-14T11:30:41.552 DEBUG:teuthology.orchestra.run.smithi017:> rm -fr /tmp/tmp.RNNru7US7m 2023-07-14T11:30:41.568 DEBUG:teuthology.orchestra.run.smithi017:> sudo /bin/ls /boot/loader/entries || true 2023-07-14T11:30:41.634 INFO:teuthology.orchestra.run.smithi017.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-07-14T11:30:41.635 INFO:teuthology.orchestra.run.smithi017.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-07-14T11:30:41.635 INFO:teuthology.orchestra.run.smithi017.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc1-ga440c79eee02.conf 2023-07-14T11:30:41.636 DEBUG:teuthology.orchestra.run.smithi017:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc1-ga440c79eee02 2023-07-14T11:30:41.796 DEBUG:teuthology.orchestra.run.smithi017:> sudo shutdown -r now 2023-07-14T11:31:11.799 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-07-14T11:31:11.800 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi017.front.sepia.ceph.com' 2023-07-14T11:31:11.801 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T11:32:11.803 DEBUG:teuthology.orchestra.remote:timed out 2023-07-14T11:32:20.806 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi017.front.sepia.ceph.com' 2023-07-14T11:32:20.807 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T11:32:22.153 DEBUG:teuthology.orchestra.run.smithi017:> true 2023-07-14T11:32:22.620 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi017.front.sepia.ceph.com' 2023-07-14T11:32:22.621 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.5.0-rc1-ga440c79eee02"... 2023-07-14T11:32:22.621 DEBUG:teuthology.orchestra.run.smithi017:> uname -r 2023-07-14T11:32:22.664 INFO:teuthology.orchestra.run.smithi017.stdout:6.5.0-rc1-ga440c79eee02 2023-07-14T11:32:22.665 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc1-ga440c79eee02 vs 6.5.0-rc1-ga440c79eee02 2023-07-14T11:32:22.665 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-07-14T11:32:22.665 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-07-14T11:32:23.666 DEBUG:teuthology.parallel:result is None 2023-07-14T11:32:23.667 INFO:teuthology.run_tasks:Running task internal.base... 2023-07-14T11:32:23.675 INFO:teuthology.task.internal:Creating test directory... 2023-07-14T11:32:23.676 DEBUG:teuthology.orchestra.run.smithi017:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-14T11:32:23.681 DEBUG:teuthology.orchestra.run.smithi093:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-14T11:32:23.684 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-14T11:32:23.724 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-07-14T11:32:23.733 INFO:teuthology.run_tasks:Running task internal.archive... 2023-07-14T11:32:23.740 INFO:teuthology.task.internal:Creating archive directory... 2023-07-14T11:32:23.741 DEBUG:teuthology.orchestra.run.smithi017:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-14T11:32:23.745 DEBUG:teuthology.orchestra.run.smithi093:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-14T11:32:23.748 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-14T11:32:23.795 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-07-14T11:32:23.804 INFO:teuthology.task.internal:Enabling coredump saving... 2023-07-14T11:32:23.805 DEBUG:teuthology.orchestra.run.smithi093:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-14T11:32:23.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T11:32:23.840 DEBUG:teuthology.orchestra.run.smithi097:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-14T11:32:23.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T11:32:23.859 DEBUG:teuthology.orchestra.run.smithi017:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-14T11:32:23.865 DEBUG:teuthology.orchestra.run.smithi093:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-14T11:32:23.883 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-14T11:32:23.936 INFO:teuthology.orchestra.run.smithi093.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-14T11:32:23.951 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-14T11:32:23.969 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-14T11:32:23.976 INFO:teuthology.orchestra.run.smithi093.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-14T11:32:24.090 INFO:teuthology.orchestra.run.smithi017.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-14T11:32:24.128 INFO:teuthology.orchestra.run.smithi017.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-14T11:32:24.131 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-07-14T11:32:24.139 INFO:teuthology.task.internal:Configuring sudo... 2023-07-14T11:32:24.139 DEBUG:teuthology.orchestra.run.smithi017:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-14T11:32:24.145 DEBUG:teuthology.orchestra.run.smithi093:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-14T11:32:24.147 DEBUG:teuthology.orchestra.run.smithi097:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-14T11:32:24.210 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-07-14T11:32:24.219 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-07-14T11:32:24.220 DEBUG:teuthology.orchestra.run.smithi017:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-14T11:32:24.224 DEBUG:teuthology.orchestra.run.smithi093:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-14T11:32:24.226 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-14T11:32:24.262 DEBUG:teuthology.orchestra.run.smithi017:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-14T11:32:24.375 DEBUG:teuthology.orchestra.run.smithi017:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-14T11:32:24.456 DEBUG:teuthology.orchestra.run.smithi017:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-14T11:32:24.530 DEBUG:teuthology.orchestra.run.smithi017:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-14T11:32:24.603 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2023-07-14T11:32:24.603 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-14T11:32:24.672 DEBUG:teuthology.orchestra.run.smithi093:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-14T11:32:24.722 DEBUG:teuthology.orchestra.run.smithi093:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-14T11:32:24.788 DEBUG:teuthology.orchestra.run.smithi093:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-14T11:32:24.838 DEBUG:teuthology.orchestra.run.smithi093:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-14T11:32:24.883 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-07-14T11:32:24.883 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-14T11:32:24.957 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-14T11:32:25.009 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-14T11:32:25.065 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-14T11:32:25.154 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-14T11:32:25.198 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-14T11:32:25.198 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-14T11:32:25.268 DEBUG:teuthology.orchestra.run.smithi017:> sudo service rsyslog restart 2023-07-14T11:32:25.273 DEBUG:teuthology.orchestra.run.smithi093:> sudo service rsyslog restart 2023-07-14T11:32:25.275 DEBUG:teuthology.orchestra.run.smithi097:> sudo service rsyslog restart 2023-07-14T11:32:25.334 INFO:teuthology.orchestra.run.smithi093.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-14T11:32:25.395 INFO:teuthology.orchestra.run.smithi097.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-14T11:32:25.435 INFO:teuthology.orchestra.run.smithi017.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-14T11:32:25.825 INFO:teuthology.run_tasks:Running task internal.timer... 2023-07-14T11:32:25.837 INFO:teuthology.task.internal:Starting timer... 2023-07-14T11:32:25.837 INFO:teuthology.run_tasks:Running task pcp... 2023-07-14T11:32:25.859 INFO:teuthology.run_tasks:Running task selinux... 2023-07-14T11:32:25.890 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-07-14T11:32:25.891 DEBUG:teuthology.orchestra.run.smithi017:> sudo service auditd rotate 2023-07-14T11:32:26.027 INFO:teuthology.orchestra.run.smithi017.stdout:Rotating logs: 2023-07-14T11:32:26.030 DEBUG:teuthology.orchestra.run.smithi093:> sudo service auditd rotate 2023-07-14T11:32:26.091 INFO:teuthology.orchestra.run.smithi093.stdout:Rotating logs: 2023-07-14T11:32:26.093 DEBUG:teuthology.orchestra.run.smithi097:> sudo service auditd rotate 2023-07-14T11:32:26.158 INFO:teuthology.orchestra.run.smithi097.stdout:Rotating logs: 2023-07-14T11:32:26.159 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-07-14T11:32:26.160 DEBUG:teuthology.orchestra.run.smithi017:> /usr/sbin/getenforce 2023-07-14T11:32:26.212 INFO:teuthology.orchestra.run.smithi017.stdout:Permissive 2023-07-14T11:32:26.212 DEBUG:teuthology.orchestra.run.smithi093:> /usr/sbin/getenforce 2023-07-14T11:32:26.236 INFO:teuthology.orchestra.run.smithi093.stdout:Permissive 2023-07-14T11:32:26.236 DEBUG:teuthology.orchestra.run.smithi097:> /usr/sbin/getenforce 2023-07-14T11:32:26.255 INFO:teuthology.orchestra.run.smithi097.stdout:Permissive 2023-07-14T11:32:26.255 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi017.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi093.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi097.front.sepia.ceph.com': 'permissive'} 2023-07-14T11:32:26.256 DEBUG:teuthology.orchestra.run.smithi017:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-14T11:32:26.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T11:32:26.326 DEBUG:teuthology.orchestra.run.smithi093:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-14T11:32:26.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T11:32:26.354 DEBUG:teuthology.orchestra.run.smithi097:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-14T11:32:26.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T11:32:26.381 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-07-14T11:32:26.382 DEBUG:teuthology.orchestra.run.smithi017:> sudo /usr/sbin/setenforce permissive 2023-07-14T11:32:26.451 DEBUG:teuthology.orchestra.run.smithi093:> sudo /usr/sbin/setenforce permissive 2023-07-14T11:32:26.493 DEBUG:teuthology.orchestra.run.smithi097:> sudo /usr/sbin/setenforce permissive 2023-07-14T11:32:26.549 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-07-14T11:32:26.559 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-07-14T11:32:26.560 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-07-14T11:32:26.582 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-07-14T11:32:26.583 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi017.front.sepia.ceph.com,smithi093.front.sepia.ceph.com,smithi097.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-07-14T11:47:36.527 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi017.front.sepia.ceph.com'), Remote(name='ubuntu@smithi093.front.sepia.ceph.com'), Remote(name='ubuntu@smithi097.front.sepia.ceph.com')] 2023-07-14T11:47:36.553 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi017.front.sepia.ceph.com' 2023-07-14T11:47:36.555 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T11:47:36.643 DEBUG:teuthology.orchestra.run.smithi017:> true 2023-07-14T11:47:36.753 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi017.front.sepia.ceph.com' 2023-07-14T11:47:36.754 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi093.front.sepia.ceph.com' 2023-07-14T11:47:36.754 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T11:47:36.822 DEBUG:teuthology.orchestra.run.smithi093:> true 2023-07-14T11:47:36.897 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi093.front.sepia.ceph.com' 2023-07-14T11:47:36.897 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2023-07-14T11:47:36.898 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T11:47:36.968 DEBUG:teuthology.orchestra.run.smithi097:> true 2023-07-14T11:47:37.040 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi097.front.sepia.ceph.com' 2023-07-14T11:47:37.041 INFO:teuthology.run_tasks:Running task clock... 2023-07-14T11:47:37.055 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-07-14T11:47:37.056 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-14T11:47:37.057 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-14T11:47:37.063 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-14T11:47:37.064 DEBUG:teuthology.orchestra.run.smithi093:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-14T11:47:37.067 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-14T11:47:37.068 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-14T11:47:37.100 INFO:teuthology.orchestra.run.smithi093.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-14T11:47:37.115 INFO:teuthology.orchestra.run.smithi093.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-14T11:47:37.116 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-14T11:47:37.132 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-14T11:47:37.135 INFO:teuthology.orchestra.run.smithi017.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-14T11:47:37.172 INFO:teuthology.orchestra.run.smithi017.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-14T11:47:37.190 INFO:teuthology.orchestra.run.smithi093.stderr:sudo: ntpd: command not found 2023-07-14T11:47:37.202 INFO:teuthology.orchestra.run.smithi093.stdout:506 Cannot talk to daemon 2023-07-14T11:47:37.203 INFO:teuthology.orchestra.run.smithi097.stderr:sudo: ntpd: command not found 2023-07-14T11:47:37.216 INFO:teuthology.orchestra.run.smithi097.stdout:506 Cannot talk to daemon 2023-07-14T11:47:37.217 INFO:teuthology.orchestra.run.smithi093.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-14T11:47:37.230 INFO:teuthology.orchestra.run.smithi093.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-14T11:47:37.231 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-14T11:47:37.245 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-14T11:47:37.266 INFO:teuthology.orchestra.run.smithi017.stderr:sudo: ntpd: command not found 2023-07-14T11:47:37.296 INFO:teuthology.orchestra.run.smithi017.stdout:506 Cannot talk to daemon 2023-07-14T11:47:37.321 INFO:teuthology.orchestra.run.smithi097.stderr:bash: ntpq: command not found 2023-07-14T11:47:37.322 INFO:teuthology.orchestra.run.smithi093.stderr:bash: ntpq: command not found 2023-07-14T11:47:37.324 INFO:teuthology.orchestra.run.smithi097.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-14T11:47:37.325 INFO:teuthology.orchestra.run.smithi097.stdout:=============================================================================== 2023-07-14T11:47:37.325 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T11:47:37.325 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T11:47:37.325 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T11:47:37.325 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T11:47:37.327 INFO:teuthology.orchestra.run.smithi093.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-14T11:47:37.327 INFO:teuthology.orchestra.run.smithi093.stdout:=============================================================================== 2023-07-14T11:47:37.327 INFO:teuthology.orchestra.run.smithi093.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T11:47:37.327 INFO:teuthology.orchestra.run.smithi093.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T11:47:37.327 INFO:teuthology.orchestra.run.smithi093.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T11:47:37.327 INFO:teuthology.orchestra.run.smithi093.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T11:47:37.331 INFO:teuthology.orchestra.run.smithi017.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-14T11:47:37.366 INFO:teuthology.orchestra.run.smithi017.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-14T11:47:37.459 INFO:teuthology.orchestra.run.smithi017.stderr:bash: ntpq: command not found 2023-07-14T11:47:37.466 INFO:teuthology.orchestra.run.smithi017.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-14T11:47:37.466 INFO:teuthology.orchestra.run.smithi017.stdout:=============================================================================== 2023-07-14T11:47:37.466 INFO:teuthology.orchestra.run.smithi017.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T11:47:37.466 INFO:teuthology.orchestra.run.smithi017.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T11:47:37.467 INFO:teuthology.orchestra.run.smithi017.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T11:47:37.467 INFO:teuthology.orchestra.run.smithi017.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T11:47:37.467 INFO:teuthology.run_tasks:Running task pexec... 2023-07-14T11:47:37.477 INFO:teuthology.task.pexec:Executing custom commands... 2023-07-14T11:47:37.478 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi017.front.sepia.ceph.com 2023-07-14T11:47:37.478 DEBUG:teuthology.orchestra.run.smithi017:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-07-14T11:47:37.478 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi093.front.sepia.ceph.com 2023-07-14T11:47:37.479 DEBUG:teuthology.orchestra.run.smithi093:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-07-14T11:47:37.479 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi097.front.sepia.ceph.com 2023-07-14T11:47:37.479 DEBUG:teuthology.orchestra.run.smithi097:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-07-14T11:47:37.850 INFO:teuthology.orchestra.run.smithi097.stdout:Updating Subscription Management repositories. 2023-07-14T11:47:37.851 INFO:teuthology.orchestra.run.smithi093.stdout:Updating Subscription Management repositories. 2023-07-14T11:47:37.997 INFO:teuthology.orchestra.run.smithi017.stdout:Updating Subscription Management repositories. 2023-07-14T11:47:38.651 INFO:teuthology.orchestra.run.smithi093.stdout:Last metadata expiration check: 0:07:42 ago on Fri 14 Jul 2023 11:39:56 AM UTC. 2023-07-14T11:47:38.673 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:33 ago on Fri 14 Jul 2023 11:46:05 AM UTC. 2023-07-14T11:47:38.992 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:05:06 ago on Fri 14 Jul 2023 11:42:32 AM UTC. 2023-07-14T11:47:40.403 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2023-07-14T11:47:40.405 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2023-07-14T11:47:40.405 INFO:teuthology.orchestra.run.smithi097.stdout: Package Architecture Version Repository Size 2023-07-14T11:47:40.406 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2023-07-14T11:47:40.406 INFO:teuthology.orchestra.run.smithi097.stdout:Resetting modules: 2023-07-14T11:47:40.406 INFO:teuthology.orchestra.run.smithi097.stdout: container-tools 2023-07-14T11:47:40.406 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-14T11:47:40.406 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2023-07-14T11:47:40.406 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2023-07-14T11:47:40.407 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-14T11:47:40.529 INFO:teuthology.orchestra.run.smithi093.stdout:Dependencies resolved. 2023-07-14T11:47:40.530 INFO:teuthology.orchestra.run.smithi093.stdout:================================================================================ 2023-07-14T11:47:40.530 INFO:teuthology.orchestra.run.smithi093.stdout: Package Architecture Version Repository Size 2023-07-14T11:47:40.531 INFO:teuthology.orchestra.run.smithi093.stdout:================================================================================ 2023-07-14T11:47:40.531 INFO:teuthology.orchestra.run.smithi093.stdout:Resetting modules: 2023-07-14T11:47:40.531 INFO:teuthology.orchestra.run.smithi093.stdout: container-tools 2023-07-14T11:47:40.531 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-14T11:47:40.531 INFO:teuthology.orchestra.run.smithi093.stdout:Transaction Summary 2023-07-14T11:47:40.531 INFO:teuthology.orchestra.run.smithi093.stdout:================================================================================ 2023-07-14T11:47:40.531 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-14T11:47:40.980 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2023-07-14T11:47:40.982 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-07-14T11:47:40.982 INFO:teuthology.orchestra.run.smithi017.stdout: Package Architecture Version Repository Size 2023-07-14T11:47:40.982 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-07-14T11:47:40.982 INFO:teuthology.orchestra.run.smithi017.stdout:Resetting modules: 2023-07-14T11:47:40.982 INFO:teuthology.orchestra.run.smithi017.stdout: container-tools 2023-07-14T11:47:40.983 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:47:40.983 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction Summary 2023-07-14T11:47:40.983 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-07-14T11:47:40.983 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:47:41.390 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:35 ago on Fri 14 Jul 2023 11:46:05 AM UTC. 2023-07-14T11:47:41.501 INFO:teuthology.orchestra.run.smithi093.stdout:Last metadata expiration check: 0:07:45 ago on Fri 14 Jul 2023 11:39:56 AM UTC. 2023-07-14T11:47:42.268 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:05:09 ago on Fri 14 Jul 2023 11:42:32 AM UTC. 2023-07-14T11:47:44.079 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2023-07-14T11:47:44.268 INFO:teuthology.orchestra.run.smithi093.stdout:Complete! 2023-07-14T11:47:44.553 INFO:teuthology.orchestra.run.smithi097.stdout:Updating Subscription Management repositories. 2023-07-14T11:47:44.735 INFO:teuthology.orchestra.run.smithi093.stdout:Updating Subscription Management repositories. 2023-07-14T11:47:44.831 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2023-07-14T11:47:45.369 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:39 ago on Fri 14 Jul 2023 11:46:05 AM UTC. 2023-07-14T11:47:45.451 INFO:teuthology.orchestra.run.smithi017.stdout:Updating Subscription Management repositories. 2023-07-14T11:47:45.526 INFO:teuthology.orchestra.run.smithi093.stdout:Last metadata expiration check: 0:07:49 ago on Fri 14 Jul 2023 11:39:56 AM UTC. 2023-07-14T11:47:46.523 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:05:14 ago on Fri 14 Jul 2023 11:42:32 AM UTC. 2023-07-14T11:47:47.104 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2023-07-14T11:47:47.195 INFO:teuthology.orchestra.run.smithi097.stdout:============================================================================================================ 2023-07-14T11:47:47.195 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2023-07-14T11:47:47.195 INFO:teuthology.orchestra.run.smithi097.stdout:============================================================================================================ 2023-07-14T11:47:47.196 INFO:teuthology.orchestra.run.smithi097.stdout:Installing group/module packages: 2023-07-14T11:47:47.196 INFO:teuthology.orchestra.run.smithi097.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-07-14T11:47:47.196 INFO:teuthology.orchestra.run.smithi097.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-07-14T11:47:47.196 INFO:teuthology.orchestra.run.smithi097.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-07-14T11:47:47.197 INFO:teuthology.orchestra.run.smithi097.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-07-14T11:47:47.197 INFO:teuthology.orchestra.run.smithi097.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-07-14T11:47:47.197 INFO:teuthology.orchestra.run.smithi097.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-07-14T11:47:47.197 INFO:teuthology.orchestra.run.smithi097.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-07-14T11:47:47.197 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2023-07-14T11:47:47.198 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-07-14T11:47:47.198 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-07-14T11:47:47.198 INFO:teuthology.orchestra.run.smithi097.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-07-14T11:47:47.198 INFO:teuthology.orchestra.run.smithi097.stdout:Installing module profiles: 2023-07-14T11:47:47.199 INFO:teuthology.orchestra.run.smithi097.stdout: container-tools/common 2023-07-14T11:47:47.199 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-14T11:47:47.199 INFO:teuthology.orchestra.run.smithi097.stdout:Enabling module streams: 2023-07-14T11:47:47.199 INFO:teuthology.orchestra.run.smithi097.stdout: container-tools rhel8 2023-07-14T11:47:47.199 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-14T11:47:47.200 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2023-07-14T11:47:47.200 INFO:teuthology.orchestra.run.smithi097.stdout:============================================================================================================ 2023-07-14T11:47:47.200 INFO:teuthology.orchestra.run.smithi097.stdout:Install 10 Packages 2023-07-14T11:47:47.200 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-14T11:47:47.201 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 18 M 2023-07-14T11:47:47.202 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 65 M 2023-07-14T11:47:47.202 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2023-07-14T11:47:47.262 INFO:teuthology.orchestra.run.smithi093.stdout:Dependencies resolved. 2023-07-14T11:47:47.274 INFO:teuthology.orchestra.run.smithi097.stdout:(1/10): yajl-2.1.0-10.el8.x86_64.rpm 533 kB/s | 41 kB 00:00 2023-07-14T11:47:47.299 INFO:teuthology.orchestra.run.smithi097.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 244 kB/s | 25 kB 00:00 2023-07-14T11:47:47.324 INFO:teuthology.orchestra.run.smithi097.stdout:(3/10): udica-0.2.6-4.module+el8.6.0+16771+28df 981 kB/s | 49 kB 00:00 2023-07-14T11:47:47.356 INFO:teuthology.orchestra.run.smithi093.stdout:============================================================================================================ 2023-07-14T11:47:47.356 INFO:teuthology.orchestra.run.smithi093.stdout: Package Arch Version Repository Size 2023-07-14T11:47:47.357 INFO:teuthology.orchestra.run.smithi093.stdout:============================================================================================================ 2023-07-14T11:47:47.357 INFO:teuthology.orchestra.run.smithi093.stdout:Installing group/module packages: 2023-07-14T11:47:47.357 INFO:teuthology.orchestra.run.smithi093.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-07-14T11:47:47.357 INFO:teuthology.orchestra.run.smithi093.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-07-14T11:47:47.357 INFO:teuthology.orchestra.run.smithi093.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-07-14T11:47:47.357 INFO:teuthology.orchestra.run.smithi093.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-07-14T11:47:47.357 INFO:teuthology.orchestra.run.smithi093.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-07-14T11:47:47.357 INFO:teuthology.orchestra.run.smithi093.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-07-14T11:47:47.358 INFO:teuthology.orchestra.run.smithi093.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-07-14T11:47:47.358 INFO:teuthology.orchestra.run.smithi093.stdout:Installing dependencies: 2023-07-14T11:47:47.358 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-07-14T11:47:47.358 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-07-14T11:47:47.358 INFO:teuthology.orchestra.run.smithi093.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-07-14T11:47:47.358 INFO:teuthology.orchestra.run.smithi093.stdout:Installing module profiles: 2023-07-14T11:47:47.358 INFO:teuthology.orchestra.run.smithi093.stdout: container-tools/common 2023-07-14T11:47:47.359 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-14T11:47:47.359 INFO:teuthology.orchestra.run.smithi093.stdout:Enabling module streams: 2023-07-14T11:47:47.359 INFO:teuthology.orchestra.run.smithi093.stdout: container-tools rhel8 2023-07-14T11:47:47.359 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-14T11:47:47.359 INFO:teuthology.orchestra.run.smithi093.stdout:Transaction Summary 2023-07-14T11:47:47.359 INFO:teuthology.orchestra.run.smithi093.stdout:============================================================================================================ 2023-07-14T11:47:47.359 INFO:teuthology.orchestra.run.smithi093.stdout:Install 10 Packages 2023-07-14T11:47:47.359 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-14T11:47:47.360 INFO:teuthology.orchestra.run.smithi093.stdout:Total download size: 18 M 2023-07-14T11:47:47.360 INFO:teuthology.orchestra.run.smithi093.stdout:Installed size: 65 M 2023-07-14T11:47:47.360 INFO:teuthology.orchestra.run.smithi093.stdout:Downloading Packages: 2023-07-14T11:47:47.383 INFO:teuthology.orchestra.run.smithi097.stdout:(4/10): python3-pyxdg-0.25-16.el8.noarch.rpm 505 kB/s | 94 kB 00:00 2023-07-14T11:47:47.417 INFO:teuthology.orchestra.run.smithi097.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 4.4 MB/s | 524 kB 00:00 2023-07-14T11:47:47.433 INFO:teuthology.orchestra.run.smithi093.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 1.2 MB/s | 94 kB 00:00 2023-07-14T11:47:47.442 INFO:teuthology.orchestra.run.smithi097.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-07-14T11:47:47.458 INFO:teuthology.orchestra.run.smithi093.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 249 kB/s | 25 kB 00:00 2023-07-14T11:47:47.467 INFO:teuthology.orchestra.run.smithi097.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.1 MB/s | 213 kB 00:00 2023-07-14T11:47:47.484 INFO:teuthology.orchestra.run.smithi093.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 326 kB/s | 41 kB 00:00 2023-07-14T11:47:47.528 INFO:teuthology.orchestra.run.smithi093.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 525 kB/s | 49 kB 00:00 2023-07-14T11:47:47.560 INFO:teuthology.orchestra.run.smithi097.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 24 MB/s | 2.2 MB 00:00 2023-07-14T11:47:47.567 INFO:teuthology.orchestra.run.smithi093.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 4.7 MB/s | 524 kB 00:00 2023-07-14T11:47:47.593 INFO:teuthology.orchestra.run.smithi093.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.2 MB/s | 149 kB 00:00 2023-07-14T11:47:47.618 INFO:teuthology.orchestra.run.smithi093.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.1 MB/s | 213 kB 00:00 2023-07-14T11:47:47.668 INFO:teuthology.orchestra.run.smithi097.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 31 MB/s | 7.0 MB 00:00 2023-07-14T11:47:47.710 INFO:teuthology.orchestra.run.smithi093.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 24 MB/s | 2.2 MB 00:00 2023-07-14T11:47:47.760 INFO:teuthology.orchestra.run.smithi097.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 18 MB/s | 8.0 MB 00:00 2023-07-14T11:47:47.760 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2023-07-14T11:47:47.761 INFO:teuthology.orchestra.run.smithi097.stdout:Total 32 MB/s | 18 MB 00:00 2023-07-14T11:47:47.827 INFO:teuthology.orchestra.run.smithi093.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 30 MB/s | 7.0 MB 00:00 2023-07-14T11:47:47.919 INFO:teuthology.orchestra.run.smithi093.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 18 MB/s | 8.0 MB 00:00 2023-07-14T11:47:47.920 INFO:teuthology.orchestra.run.smithi093.stdout:-------------------------------------------------------------------------------- 2023-07-14T11:47:47.921 INFO:teuthology.orchestra.run.smithi093.stdout:Total 32 MB/s | 18 MB 00:00 2023-07-14T11:47:47.959 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2023-07-14T11:47:47.978 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2023-07-14T11:47:47.978 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2023-07-14T11:47:48.119 INFO:teuthology.orchestra.run.smithi093.stdout:Running transaction check 2023-07-14T11:47:48.138 INFO:teuthology.orchestra.run.smithi093.stdout:Transaction check succeeded. 2023-07-14T11:47:48.138 INFO:teuthology.orchestra.run.smithi093.stdout:Running transaction test 2023-07-14T11:47:48.161 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2023-07-14T11:47:48.165 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2023-07-14T11:47:48.320 INFO:teuthology.orchestra.run.smithi093.stdout:Transaction test succeeded. 2023-07-14T11:47:48.324 INFO:teuthology.orchestra.run.smithi093.stdout:Running transaction 2023-07-14T11:47:48.665 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2023-07-14T11:47:48.667 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2023-07-14T11:47:48.773 INFO:teuthology.orchestra.run.smithi017.stdout:============================================================================================================ 2023-07-14T11:47:48.773 INFO:teuthology.orchestra.run.smithi017.stdout: Package Arch Version Repository Size 2023-07-14T11:47:48.774 INFO:teuthology.orchestra.run.smithi017.stdout:============================================================================================================ 2023-07-14T11:47:48.774 INFO:teuthology.orchestra.run.smithi017.stdout:Installing group/module packages: 2023-07-14T11:47:48.774 INFO:teuthology.orchestra.run.smithi017.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-07-14T11:47:48.774 INFO:teuthology.orchestra.run.smithi017.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-07-14T11:47:48.774 INFO:teuthology.orchestra.run.smithi017.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-07-14T11:47:48.774 INFO:teuthology.orchestra.run.smithi017.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-07-14T11:47:48.774 INFO:teuthology.orchestra.run.smithi017.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-07-14T11:47:48.774 INFO:teuthology.orchestra.run.smithi017.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-07-14T11:47:48.775 INFO:teuthology.orchestra.run.smithi017.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-07-14T11:47:48.775 INFO:teuthology.orchestra.run.smithi017.stdout:Installing dependencies: 2023-07-14T11:47:48.775 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-07-14T11:47:48.775 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-07-14T11:47:48.775 INFO:teuthology.orchestra.run.smithi017.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-07-14T11:47:48.775 INFO:teuthology.orchestra.run.smithi017.stdout:Installing module profiles: 2023-07-14T11:47:48.775 INFO:teuthology.orchestra.run.smithi017.stdout: container-tools/common 2023-07-14T11:47:48.776 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:47:48.776 INFO:teuthology.orchestra.run.smithi017.stdout:Enabling module streams: 2023-07-14T11:47:48.776 INFO:teuthology.orchestra.run.smithi017.stdout: container-tools rhel8 2023-07-14T11:47:48.776 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:47:48.776 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction Summary 2023-07-14T11:47:48.776 INFO:teuthology.orchestra.run.smithi017.stdout:============================================================================================================ 2023-07-14T11:47:48.776 INFO:teuthology.orchestra.run.smithi017.stdout:Install 10 Packages 2023-07-14T11:47:48.776 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:47:48.777 INFO:teuthology.orchestra.run.smithi017.stdout:Total download size: 18 M 2023-07-14T11:47:48.777 INFO:teuthology.orchestra.run.smithi017.stdout:Installed size: 65 M 2023-07-14T11:47:48.777 INFO:teuthology.orchestra.run.smithi017.stdout:Downloading Packages: 2023-07-14T11:47:48.861 INFO:teuthology.orchestra.run.smithi093.stdout: Preparing : 1/1 2023-07-14T11:47:48.878 INFO:teuthology.orchestra.run.smithi017.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 931 kB/s | 94 kB 00:00 2023-07-14T11:47:48.888 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-07-14T11:47:48.908 INFO:teuthology.orchestra.run.smithi017.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 189 kB/s | 25 kB 00:00 2023-07-14T11:47:48.933 INFO:teuthology.orchestra.run.smithi017.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 260 kB/s | 41 kB 00:00 2023-07-14T11:47:48.959 INFO:teuthology.orchestra.run.smithi017.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 610 kB/s | 49 kB 00:00 2023-07-14T11:47:49.001 INFO:teuthology.orchestra.run.smithi017.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 5.6 MB/s | 524 kB 00:00 2023-07-14T11:47:49.030 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-07-14T11:47:49.034 INFO:teuthology.orchestra.run.smithi017.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 1.9 MB/s | 149 kB 00:00 2023-07-14T11:47:49.054 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-14T11:47:49.068 INFO:teuthology.orchestra.run.smithi017.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 3.1 MB/s | 213 kB 00:00 2023-07-14T11:47:49.202 INFO:teuthology.orchestra.run.smithi017.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 16 MB/s | 2.2 MB 00:00 2023-07-14T11:47:49.229 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-14T11:47:49.248 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-07-14T11:47:49.369 INFO:teuthology.orchestra.run.smithi017.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 21 MB/s | 7.0 MB 00:00 2023-07-14T11:47:49.423 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-07-14T11:47:49.469 INFO:teuthology.orchestra.run.smithi017.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 15 MB/s | 8.0 MB 00:00 2023-07-14T11:47:49.469 INFO:teuthology.orchestra.run.smithi017.stdout:-------------------------------------------------------------------------------- 2023-07-14T11:47:49.470 INFO:teuthology.orchestra.run.smithi017.stdout:Total 26 MB/s | 18 MB 00:00 2023-07-14T11:47:49.476 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-07-14T11:47:49.627 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-07-14T11:47:49.811 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction check 2023-07-14T11:47:49.833 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-07-14T11:47:49.835 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction check succeeded. 2023-07-14T11:47:49.835 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction test 2023-07-14T11:47:49.983 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-07-14T11:47:50.335 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction test succeeded. 2023-07-14T11:47:50.340 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction 2023-07-14T11:47:50.608 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-07-14T11:47:50.762 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-07-14T11:47:50.949 INFO:teuthology.orchestra.run.smithi017.stdout: Preparing : 1/1 2023-07-14T11:47:51.255 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-07-14T11:47:51.465 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-14T11:47:51.486 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-07-14T11:47:51.670 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-07-14T11:47:51.687 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-07-14T11:47:51.738 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-07-14T11:47:51.890 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-07-14T11:47:51.912 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-07-14T11:47:52.037 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-07-14T11:47:52.106 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-14T11:47:52.222 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-14T11:47:52.267 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-07-14T11:47:52.371 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-14T11:47:52.372 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-07-14T11:47:52.372 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-14T11:47:52.372 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-07-14T11:47:52.372 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-07-14T11:47:52.372 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-07-14T11:47:52.373 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-07-14T11:47:52.373 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-07-14T11:47:52.373 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-07-14T11:47:52.373 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-07-14T11:47:52.472 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-14T11:47:52.472 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-07-14T11:47:52.473 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-14T11:47:52.473 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-07-14T11:47:52.473 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-07-14T11:47:52.473 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-07-14T11:47:52.473 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-07-14T11:47:52.473 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-07-14T11:47:52.473 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-07-14T11:47:52.473 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-07-14T11:47:52.669 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-07-14T11:47:53.565 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-07-14T11:47:53.588 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-07-14T11:47:53.588 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:48 ago on Fri 14 Jul 2023 11:46:05 AM UTC. 2023-07-14T11:47:53.700 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-07-14T11:47:53.700 INFO:teuthology.orchestra.run.smithi093.stdout:Last metadata expiration check: 0:07:57 ago on Fri 14 Jul 2023 11:39:56 AM UTC. 2023-07-14T11:47:54.551 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-07-14T11:47:54.845 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-07-14T11:47:55.055 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-07-14T11:47:55.243 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-14T11:47:55.636 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-14T11:47:55.636 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-07-14T11:47:55.636 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-14T11:47:55.636 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-07-14T11:47:55.637 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-07-14T11:47:55.637 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-07-14T11:47:55.637 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-07-14T11:47:55.637 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-07-14T11:47:55.637 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-07-14T11:47:55.637 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-07-14T11:47:57.088 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-07-14T11:47:57.088 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:05:24 ago on Fri 14 Jul 2023 11:42:32 AM UTC. 2023-07-14T11:47:57.121 INFO:teuthology.orchestra.run.smithi097.stdout:Installed products updated. 2023-07-14T11:47:57.125 INFO:teuthology.orchestra.run.smithi093.stdout:Installed products updated. 2023-07-14T11:47:57.324 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-14T11:47:57.324 INFO:teuthology.orchestra.run.smithi093.stdout:Installed: 2023-07-14T11:47:57.325 INFO:teuthology.orchestra.run.smithi093.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T11:47:57.325 INFO:teuthology.orchestra.run.smithi093.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-07-14T11:47:57.325 INFO:teuthology.orchestra.run.smithi093.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T11:47:57.325 INFO:teuthology.orchestra.run.smithi093.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-07-14T11:47:57.325 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-07-14T11:47:57.325 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-07-14T11:47:57.325 INFO:teuthology.orchestra.run.smithi093.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T11:47:57.326 INFO:teuthology.orchestra.run.smithi093.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T11:47:57.326 INFO:teuthology.orchestra.run.smithi093.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-07-14T11:47:57.326 INFO:teuthology.orchestra.run.smithi093.stdout: yajl-2.1.0-10.el8.x86_64 2023-07-14T11:47:57.327 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-14T11:47:57.327 INFO:teuthology.orchestra.run.smithi093.stdout:Complete! 2023-07-14T11:47:57.387 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-14T11:47:57.387 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2023-07-14T11:47:57.388 INFO:teuthology.orchestra.run.smithi097.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T11:47:57.388 INFO:teuthology.orchestra.run.smithi097.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-07-14T11:47:57.388 INFO:teuthology.orchestra.run.smithi097.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T11:47:57.388 INFO:teuthology.orchestra.run.smithi097.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-07-14T11:47:57.388 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-07-14T11:47:57.388 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-07-14T11:47:57.389 INFO:teuthology.orchestra.run.smithi097.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T11:47:57.389 INFO:teuthology.orchestra.run.smithi097.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T11:47:57.389 INFO:teuthology.orchestra.run.smithi097.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-07-14T11:47:57.389 INFO:teuthology.orchestra.run.smithi097.stdout: yajl-2.1.0-10.el8.x86_64 2023-07-14T11:47:57.389 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-14T11:47:57.389 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2023-07-14T11:47:57.479 DEBUG:teuthology.parallel:result is None 2023-07-14T11:47:57.558 DEBUG:teuthology.parallel:result is None 2023-07-14T11:47:59.989 INFO:teuthology.orchestra.run.smithi017.stdout:Installed products updated. 2023-07-14T11:48:00.207 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:48:00.207 INFO:teuthology.orchestra.run.smithi017.stdout:Installed: 2023-07-14T11:48:00.207 INFO:teuthology.orchestra.run.smithi017.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T11:48:00.208 INFO:teuthology.orchestra.run.smithi017.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-07-14T11:48:00.208 INFO:teuthology.orchestra.run.smithi017.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T11:48:00.208 INFO:teuthology.orchestra.run.smithi017.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-07-14T11:48:00.208 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-07-14T11:48:00.208 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-07-14T11:48:00.208 INFO:teuthology.orchestra.run.smithi017.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T11:48:00.208 INFO:teuthology.orchestra.run.smithi017.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T11:48:00.209 INFO:teuthology.orchestra.run.smithi017.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-07-14T11:48:00.209 INFO:teuthology.orchestra.run.smithi017.stdout: yajl-2.1.0-10.el8.x86_64 2023-07-14T11:48:00.209 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:48:00.209 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2023-07-14T11:48:00.434 DEBUG:teuthology.parallel:result is None 2023-07-14T11:48:00.435 INFO:teuthology.run_tasks:Running task install... 2023-07-14T11:48:00.444 DEBUG:teuthology.task.install:project ceph 2023-07-14T11:48:00.445 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3fc0089cc1082313a98853d48b892acdc1385b8b'}} 2023-07-14T11:48:00.445 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '3fc0089cc1082313a98853d48b892acdc1385b8b'} 2023-07-14T11:48:00.445 INFO:teuthology.task.install:Using flavor: default 2023-07-14T11:48:00.452 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-07-14T11:48:00.452 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-07-14T11:48:00.453 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3fc0089cc1082313a98853d48b892acdc1385b8b', 'tag': None, 'wait_for_package': False} 2023-07-14T11:48:00.453 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-14T11:48:00.455 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3fc0089cc1082313a98853d48b892acdc1385b8b', 'tag': None, 'wait_for_package': False} 2023-07-14T11:48:00.456 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-14T11:48:00.457 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3fc0089cc1082313a98853d48b892acdc1385b8b', 'tag': None, 'wait_for_package': False} 2023-07-14T11:48:00.457 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-14T11:48:00.866 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/ 2023-07-14T11:48:00.866 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4757.g3fc0089c 2023-07-14T11:48:00.870 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/ 2023-07-14T11:48:00.871 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4757.g3fc0089c 2023-07-14T11:48:00.877 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/ 2023-07-14T11:48:00.877 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4757.g3fc0089c 2023-07-14T11:48:01.199 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-14T11:48:01.199 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-07-14T11:48:01.200 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-14T11:48:01.209 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-14T11:48:01.209 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-14T11:48:01.209 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-14T11:48:01.213 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-14T11:48:01.213 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2023-07-14T11:48:01.213 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-14T11:48:01.226 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-14T11:48:01.226 DEBUG:teuthology.orchestra.run.smithi093:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3fc0089cc1082313a98853d48b892acdc1385b8b/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-14T11:48:01.236 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-14T11:48:01.236 DEBUG:teuthology.orchestra.run.smithi097:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3fc0089cc1082313a98853d48b892acdc1385b8b/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-14T11:48:01.283 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-14T11:48:01.284 DEBUG:teuthology.orchestra.run.smithi017:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3fc0089cc1082313a98853d48b892acdc1385b8b/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-14T11:48:01.295 DEBUG:teuthology.orchestra.run.smithi093:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-14T11:48:01.303 DEBUG:teuthology.orchestra.run.smithi097:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-14T11:48:01.373 DEBUG:teuthology.orchestra.run.smithi093:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-14T11:48:01.381 DEBUG:teuthology.orchestra.run.smithi097:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-14T11:48:01.395 DEBUG:teuthology.orchestra.run.smithi017:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-14T11:48:01.440 INFO:teuthology.orchestra.run.smithi093.stdout:check_obsoletes = 1 2023-07-14T11:48:01.442 DEBUG:teuthology.orchestra.run.smithi093:> sudo yum clean all 2023-07-14T11:48:01.447 INFO:teuthology.orchestra.run.smithi097.stdout:check_obsoletes = 1 2023-07-14T11:48:01.448 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum clean all 2023-07-14T11:48:01.494 DEBUG:teuthology.orchestra.run.smithi017:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-14T11:48:01.564 INFO:teuthology.orchestra.run.smithi017.stdout:check_obsoletes = 1 2023-07-14T11:48:01.567 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum clean all 2023-07-14T11:48:01.843 INFO:teuthology.orchestra.run.smithi097.stdout:Updating Subscription Management repositories. 2023-07-14T11:48:01.848 INFO:teuthology.orchestra.run.smithi093.stdout:Updating Subscription Management repositories. 2023-07-14T11:48:02.104 INFO:teuthology.orchestra.run.smithi017.stdout:Updating Subscription Management repositories. 2023-07-14T11:48:02.258 INFO:teuthology.orchestra.run.smithi097.stdout:44 files removed 2023-07-14T11:48:02.266 INFO:teuthology.orchestra.run.smithi093.stdout:44 files removed 2023-07-14T11:48:02.312 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-14T11:48:02.323 DEBUG:teuthology.orchestra.run.smithi093:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-14T11:48:02.585 INFO:teuthology.orchestra.run.smithi017.stdout:44 files removed 2023-07-14T11:48:02.650 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-14T11:48:02.668 INFO:teuthology.orchestra.run.smithi093.stdout:Updating Subscription Management repositories. 2023-07-14T11:48:02.738 INFO:teuthology.orchestra.run.smithi097.stdout:Updating Subscription Management repositories. 2023-07-14T11:48:03.140 INFO:teuthology.orchestra.run.smithi017.stdout:Updating Subscription Management repositories. 2023-07-14T11:48:04.014 INFO:teuthology.orchestra.run.smithi093.stdout:ceph packages for x86_64 89 kB/s | 81 kB 00:00 2023-07-14T11:48:04.112 INFO:teuthology.orchestra.run.smithi097.stdout:ceph packages for x86_64 87 kB/s | 81 kB 00:00 2023-07-14T11:48:04.500 INFO:teuthology.orchestra.run.smithi017.stdout:ceph packages for x86_64 93 kB/s | 81 kB 00:00 2023-07-14T11:48:04.682 INFO:teuthology.orchestra.run.smithi093.stdout:ceph noarch packages 15 kB/s | 9.4 kB 00:00 2023-07-14T11:48:04.819 INFO:teuthology.orchestra.run.smithi097.stdout:ceph noarch packages 14 kB/s | 9.4 kB 00:00 2023-07-14T11:48:05.254 INFO:teuthology.orchestra.run.smithi017.stdout:ceph noarch packages 13 kB/s | 9.4 kB 00:00 2023-07-14T11:48:05.502 INFO:teuthology.orchestra.run.smithi093.stdout:ceph source packages 1.9 kB/s | 1.5 kB 00:00 2023-07-14T11:48:05.777 INFO:teuthology.orchestra.run.smithi097.stdout:ceph source packages 1.6 kB/s | 1.5 kB 00:00 2023-07-14T11:48:06.252 INFO:teuthology.orchestra.run.smithi017.stdout:ceph source packages 1.5 kB/s | 1.5 kB 00:00 2023-07-14T11:48:06.264 INFO:teuthology.orchestra.run.smithi097.stdout:Copr repo for python3-asyncssh owned by ceph 7.4 kB/s | 3.5 kB 00:00 2023-07-14T11:48:06.324 INFO:teuthology.orchestra.run.smithi093.stdout:Copr repo for python3-asyncssh owned by ceph 4.3 kB/s | 3.5 kB 00:00 2023-07-14T11:48:06.770 INFO:teuthology.orchestra.run.smithi017.stdout:Copr repo for python3-asyncssh owned by ceph 7.0 kB/s | 3.5 kB 00:00 2023-07-14T11:48:08.339 INFO:teuthology.orchestra.run.smithi097.stdout:Extra Packages for Enterprise Linux 7.6 MB/s | 16 MB 00:02 2023-07-14T11:48:11.747 INFO:teuthology.orchestra.run.smithi097.stdout:lab-extras 410 kB/s | 24 kB 00:00 2023-07-14T11:48:12.674 INFO:teuthology.orchestra.run.smithi097.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 55 MB/s | 47 MB 00:00 2023-07-14T11:48:20.508 INFO:teuthology.orchestra.run.smithi097.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 65 MB/s | 53 MB 00:00 2023-07-14T11:48:21.672 INFO:teuthology.orchestra.run.smithi093.stdout:Extra Packages for Enterprise Linux 1.0 MB/s | 16 MB 00:15 2023-07-14T11:48:21.823 INFO:teuthology.orchestra.run.smithi017.stdout:Extra Packages for Enterprise Linux 1.0 MB/s | 16 MB 00:15 2023-07-14T11:48:25.123 INFO:teuthology.orchestra.run.smithi093.stdout:lab-extras 343 kB/s | 24 kB 00:00 2023-07-14T11:48:26.016 INFO:teuthology.orchestra.run.smithi093.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 64 MB/s | 53 MB 00:00 2023-07-14T11:48:26.165 INFO:teuthology.orchestra.run.smithi017.stdout:lab-extras 282 kB/s | 24 kB 00:00 2023-07-14T11:48:27.492 INFO:teuthology.orchestra.run.smithi017.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 43 MB/s | 53 MB 00:01 2023-07-14T11:48:31.619 INFO:teuthology.orchestra.run.smithi097.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-14T11:48:31.619 INFO:teuthology.orchestra.run.smithi097.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-14T11:48:31.624 INFO:teuthology.orchestra.run.smithi097.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-14T11:48:31.625 INFO:teuthology.orchestra.run.smithi097.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-14T11:48:31.626 INFO:teuthology.orchestra.run.smithi097.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-14T11:48:31.627 INFO:teuthology.orchestra.run.smithi097.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-14T11:48:31.629 INFO:teuthology.orchestra.run.smithi097.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-14T11:48:31.633 INFO:teuthology.orchestra.run.smithi097.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-14T11:48:31.634 INFO:teuthology.orchestra.run.smithi097.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-14T11:48:31.763 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2023-07-14T11:48:31.775 INFO:teuthology.orchestra.run.smithi097.stdout:=========================================================================================================================== 2023-07-14T11:48:31.775 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2023-07-14T11:48:31.776 INFO:teuthology.orchestra.run.smithi097.stdout:=========================================================================================================================== 2023-07-14T11:48:31.776 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2023-07-14T11:48:31.776 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-14T11:48:31.776 INFO:teuthology.orchestra.run.smithi097.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-14T11:48:31.776 INFO:teuthology.orchestra.run.smithi097.stdout: ceph x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 6.4 k 2023-07-14T11:48:31.776 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 5.4 M 2023-07-14T11:48:31.777 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-fuse x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 848 k 2023-07-14T11:48:31.777 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 165 k 2023-07-14T11:48:31.777 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 1.5 M 2023-07-14T11:48:31.777 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 129 k 2023-07-14T11:48:31.777 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 1.7 M 2023-07-14T11:48:31.777 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 7.4 M 2023-07-14T11:48:31.777 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-rook noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 47 k 2023-07-14T11:48:31.777 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-radosgw x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 13 M 2023-07-14T11:48:31.778 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-test x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 48 M 2023-07-14T11:48:31.778 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-volume noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 258 k 2023-07-14T11:48:31.778 INFO:teuthology.orchestra.run.smithi097.stdout: cephadm noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 166 k 2023-07-14T11:48:31.778 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-mirror x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 258 k 2023-07-14T11:48:31.778 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-top noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 18 k 2023-07-14T11:48:31.778 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-14T11:48:31.778 INFO:teuthology.orchestra.run.smithi097.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-14T11:48:31.779 INFO:teuthology.orchestra.run.smithi097.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-14T11:48:31.779 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs-devel x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 31 k 2023-07-14T11:48:31.779 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs2 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 771 k 2023-07-14T11:48:31.779 INFO:teuthology.orchestra.run.smithi097.stdout: librados-devel x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 139 k 2023-07-14T11:48:31.779 INFO:teuthology.orchestra.run.smithi097.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-14T11:48:31.779 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-14T11:48:31.779 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-14T11:48:31.780 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-14T11:48:31.780 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-14T11:48:31.780 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-14T11:48:31.780 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cephfs x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 197 k 2023-07-14T11:48:31.780 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rados x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 371 k 2023-07-14T11:48:31.780 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rbd x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 364 k 2023-07-14T11:48:31.780 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rgw x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 107 k 2023-07-14T11:48:31.780 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-fuse x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 94 k 2023-07-14T11:48:31.781 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-mirror x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 3.4 M 2023-07-14T11:48:31.781 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-nbd x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 179 k 2023-07-14T11:48:31.781 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-14T11:48:31.781 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrading: 2023-07-14T11:48:31.781 INFO:teuthology.orchestra.run.smithi097.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-14T11:48:31.781 INFO:teuthology.orchestra.run.smithi097.stdout: librados2 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 3.7 M 2023-07-14T11:48:31.781 INFO:teuthology.orchestra.run.smithi097.stdout: librbd1 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 3.6 M 2023-07-14T11:48:31.782 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2023-07-14T11:48:31.782 INFO:teuthology.orchestra.run.smithi097.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-14T11:48:31.782 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-common x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 22 M 2023-07-14T11:48:31.782 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 23 k 2023-07-14T11:48:31.782 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mds x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 2.3 M 2023-07-14T11:48:31.782 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 240 k 2023-07-14T11:48:31.782 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mon x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 4.6 M 2023-07-14T11:48:31.782 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-osd x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 18 M 2023-07-14T11:48:31.783 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 15 k 2023-07-14T11:48:31.783 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-selinux x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 24 k 2023-07-14T11:48:31.783 INFO:teuthology.orchestra.run.smithi097.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-14T11:48:31.783 INFO:teuthology.orchestra.run.smithi097.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-14T11:48:31.783 INFO:teuthology.orchestra.run.smithi097.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-14T11:48:31.783 INFO:teuthology.orchestra.run.smithi097.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-14T11:48:31.783 INFO:teuthology.orchestra.run.smithi097.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-14T11:48:31.784 INFO:teuthology.orchestra.run.smithi097.stdout: libcephsqlite x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 137 k 2023-07-14T11:48:31.784 INFO:teuthology.orchestra.run.smithi097.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-14T11:48:31.784 INFO:teuthology.orchestra.run.smithi097.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-14T11:48:31.784 INFO:teuthology.orchestra.run.smithi097.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-14T11:48:31.784 INFO:teuthology.orchestra.run.smithi097.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-14T11:48:31.784 INFO:teuthology.orchestra.run.smithi097.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-14T11:48:31.784 INFO:teuthology.orchestra.run.smithi097.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-14T11:48:31.785 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 473 k 2023-07-14T11:48:31.785 INFO:teuthology.orchestra.run.smithi097.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-14T11:48:31.785 INFO:teuthology.orchestra.run.smithi097.stdout: librgw2 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 6.7 M 2023-07-14T11:48:31.785 INFO:teuthology.orchestra.run.smithi097.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-14T11:48:31.785 INFO:teuthology.orchestra.run.smithi097.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-14T11:48:31.785 INFO:teuthology.orchestra.run.smithi097.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-14T11:48:31.785 INFO:teuthology.orchestra.run.smithi097.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-14T11:48:31.785 INFO:teuthology.orchestra.run.smithi097.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-14T11:48:31.786 INFO:teuthology.orchestra.run.smithi097.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-14T11:48:31.786 INFO:teuthology.orchestra.run.smithi097.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-14T11:48:31.786 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-14T11:48:31.786 INFO:teuthology.orchestra.run.smithi097.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-14T11:48:31.786 INFO:teuthology.orchestra.run.smithi097.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-14T11:48:31.786 INFO:teuthology.orchestra.run.smithi097.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-14T11:48:31.786 INFO:teuthology.orchestra.run.smithi097.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-14T11:48:31.786 INFO:teuthology.orchestra.run.smithi097.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-14T11:48:31.787 INFO:teuthology.orchestra.run.smithi097.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-14T11:48:31.787 INFO:teuthology.orchestra.run.smithi097.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-14T11:48:31.787 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-14T11:48:31.787 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-argparse x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 45 k 2023-07-14T11:48:31.787 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-common x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 119 k 2023-07-14T11:48:31.787 INFO:teuthology.orchestra.run.smithi097.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-14T11:48:31.787 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-14T11:48:31.787 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-14T11:48:31.788 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-14T11:48:31.788 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-14T11:48:31.788 INFO:teuthology.orchestra.run.smithi097.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-14T11:48:31.788 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-14T11:48:31.788 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-14T11:48:31.788 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-14T11:48:31.788 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-14T11:48:31.789 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-14T11:48:31.789 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-14T11:48:31.789 INFO:teuthology.orchestra.run.smithi097.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-14T11:48:31.789 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-14T11:48:31.789 INFO:teuthology.orchestra.run.smithi097.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-14T11:48:31.789 INFO:teuthology.orchestra.run.smithi097.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-14T11:48:31.789 INFO:teuthology.orchestra.run.smithi097.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-14T11:48:31.789 INFO:teuthology.orchestra.run.smithi097.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-14T11:48:31.790 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-14T11:48:31.790 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-14T11:48:31.790 INFO:teuthology.orchestra.run.smithi097.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-14T11:48:31.790 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-14T11:48:31.790 INFO:teuthology.orchestra.run.smithi097.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-14T11:48:31.790 INFO:teuthology.orchestra.run.smithi097.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-14T11:48:31.790 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-14T11:48:31.790 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-14T11:48:31.791 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-14T11:48:31.791 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-14T11:48:31.791 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-14T11:48:31.791 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-14T11:48:31.791 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-14T11:48:31.791 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-14T11:48:31.791 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-14T11:48:31.791 INFO:teuthology.orchestra.run.smithi097.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-14T11:48:31.792 INFO:teuthology.orchestra.run.smithi097.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-14T11:48:31.792 INFO:teuthology.orchestra.run.smithi097.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-14T11:48:31.792 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-14T11:48:31.792 INFO:teuthology.orchestra.run.smithi097.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-14T11:48:31.792 INFO:teuthology.orchestra.run.smithi097.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-14T11:48:31.792 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-14T11:48:31.792 INFO:teuthology.orchestra.run.smithi097.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-14T11:48:31.792 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-14T11:48:31.793 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-14T11:48:31.793 INFO:teuthology.orchestra.run.smithi097.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-14T11:48:31.793 INFO:teuthology.orchestra.run.smithi097.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-14T11:48:31.793 INFO:teuthology.orchestra.run.smithi097.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-14T11:48:31.793 INFO:teuthology.orchestra.run.smithi097.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-14T11:48:31.793 INFO:teuthology.orchestra.run.smithi097.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-14T11:48:31.794 INFO:teuthology.orchestra.run.smithi097.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-14T11:48:31.794 INFO:teuthology.orchestra.run.smithi097.stdout:Enabling module streams: 2023-07-14T11:48:31.794 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql 10 2023-07-14T11:48:31.794 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-14T11:48:31.794 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2023-07-14T11:48:31.794 INFO:teuthology.orchestra.run.smithi097.stdout:=========================================================================================================================== 2023-07-14T11:48:31.794 INFO:teuthology.orchestra.run.smithi097.stdout:Install 128 Packages 2023-07-14T11:48:31.794 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrade 3 Packages 2023-07-14T11:48:31.795 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-14T11:48:31.795 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 205 M 2023-07-14T11:48:31.795 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2023-07-14T11:48:32.584 INFO:teuthology.orchestra.run.smithi097.stdout:(1/131): ceph-18.0.0-4757.g3fc0089c.el8.x86_64. 20 kB/s | 6.4 kB 00:00 2023-07-14T11:48:33.110 INFO:teuthology.orchestra.run.smithi097.stdout:(2/131): ceph-fuse-18.0.0-4757.g3fc0089c.el8.x8 1.6 MB/s | 848 kB 00:00 2023-07-14T11:48:33.218 INFO:teuthology.orchestra.run.smithi097.stdout:(3/131): ceph-immutable-object-cache-18.0.0-475 1.5 MB/s | 165 kB 00:00 2023-07-14T11:48:33.444 INFO:teuthology.orchestra.run.smithi097.stdout:(4/131): ceph-base-18.0.0-4757.g3fc0089c.el8.x8 4.6 MB/s | 5.4 MB 00:01 2023-07-14T11:48:33.660 INFO:teuthology.orchestra.run.smithi097.stdout:(5/131): ceph-mgr-18.0.0-4757.g3fc0089c.el8.x86 7.2 MB/s | 1.5 MB 00:00 2023-07-14T11:48:33.936 INFO:teuthology.orchestra.run.smithi097.stdout:(6/131): ceph-mds-18.0.0-4757.g3fc0089c.el8.x86 3.2 MB/s | 2.3 MB 00:00 2023-07-14T11:48:34.100 INFO:teuthology.orchestra.run.smithi093.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 60 MB/s | 47 MB 00:00 2023-07-14T11:48:34.169 INFO:teuthology.orchestra.run.smithi097.stdout:(7/131): ceph-mon-18.0.0-4757.g3fc0089c.el8.x86 9.1 MB/s | 4.6 MB 00:00 2023-07-14T11:48:35.453 INFO:teuthology.orchestra.run.smithi097.stdout:(8/131): ceph-common-18.0.0-4757.g3fc0089c.el8. 6.9 MB/s | 22 MB 00:03 2023-07-14T11:48:35.561 INFO:teuthology.orchestra.run.smithi097.stdout:(9/131): ceph-selinux-18.0.0-4757.g3fc0089c.el8 225 kB/s | 24 kB 00:00 2023-07-14T11:48:35.803 INFO:teuthology.orchestra.run.smithi097.stdout:(10/131): ceph-radosgw-18.0.0-4757.g3fc0089c.el 8.3 MB/s | 13 MB 00:01 2023-07-14T11:48:35.903 INFO:teuthology.orchestra.run.smithi097.stdout:(11/131): cephfs-mirror-18.0.0-4757.g3fc0089c.e 2.5 MB/s | 258 kB 00:00 2023-07-14T11:48:36.029 INFO:teuthology.orchestra.run.smithi097.stdout:(12/131): libcephfs-devel-18.0.0-4757.g3fc0089c 248 kB/s | 31 kB 00:00 2023-07-14T11:48:36.179 INFO:teuthology.orchestra.run.smithi097.stdout:(13/131): libcephfs2-18.0.0-4757.g3fc0089c.el8. 5.0 MB/s | 771 kB 00:00 2023-07-14T11:48:36.271 INFO:teuthology.orchestra.run.smithi097.stdout:(14/131): libcephsqlite-18.0.0-4757.g3fc0089c.e 1.5 MB/s | 137 kB 00:00 2023-07-14T11:48:36.371 INFO:teuthology.orchestra.run.smithi097.stdout:(15/131): librados-devel-18.0.0-4757.g3fc0089c. 1.4 MB/s | 139 kB 00:00 2023-07-14T11:48:36.505 INFO:teuthology.orchestra.run.smithi097.stdout:(16/131): libradosstriper1-18.0.0-4757.g3fc0089 3.5 MB/s | 473 kB 00:00 2023-07-14T11:48:36.927 INFO:teuthology.orchestra.run.smithi017.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 46 MB/s | 47 MB 00:01 2023-07-14T11:48:37.455 INFO:teuthology.orchestra.run.smithi097.stdout:(17/131): librgw2-18.0.0-4757.g3fc0089c.el8.x86 7.1 MB/s | 6.7 MB 00:00 2023-07-14T11:48:37.547 INFO:teuthology.orchestra.run.smithi097.stdout:(18/131): python3-ceph-argparse-18.0.0-4757.g3f 489 kB/s | 45 kB 00:00 2023-07-14T11:48:37.647 INFO:teuthology.orchestra.run.smithi097.stdout:(19/131): python3-ceph-common-18.0.0-4757.g3fc0 1.2 MB/s | 119 kB 00:00 2023-07-14T11:48:37.873 INFO:teuthology.orchestra.run.smithi097.stdout:(20/131): ceph-osd-18.0.0-4757.g3fc0089c.el8.x8 4.5 MB/s | 18 MB 00:03 2023-07-14T11:48:37.898 INFO:teuthology.orchestra.run.smithi097.stdout:(21/131): python3-cephfs-18.0.0-4757.g3fc0089c. 786 kB/s | 197 kB 00:00 2023-07-14T11:48:38.006 INFO:teuthology.orchestra.run.smithi097.stdout:(22/131): python3-rbd-18.0.0-4757.g3fc0089c.el8 3.3 MB/s | 364 kB 00:00 2023-07-14T11:48:38.040 INFO:teuthology.orchestra.run.smithi097.stdout:(23/131): python3-rados-18.0.0-4757.g3fc0089c.e 2.2 MB/s | 371 kB 00:00 2023-07-14T11:48:38.132 INFO:teuthology.orchestra.run.smithi097.stdout:(24/131): python3-rgw-18.0.0-4757.g3fc0089c.el8 853 kB/s | 107 kB 00:00 2023-07-14T11:48:38.157 INFO:teuthology.orchestra.run.smithi097.stdout:(25/131): rbd-fuse-18.0.0-4757.g3fc0089c.el8.x8 808 kB/s | 94 kB 00:00 2023-07-14T11:48:38.249 INFO:teuthology.orchestra.run.smithi097.stdout:(26/131): rbd-nbd-18.0.0-4757.g3fc0089c.el8.x86 1.9 MB/s | 179 kB 00:00 2023-07-14T11:48:38.341 INFO:teuthology.orchestra.run.smithi097.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4757.g 252 kB/s | 23 kB 00:00 2023-07-14T11:48:38.458 INFO:teuthology.orchestra.run.smithi097.stdout:(28/131): ceph-mgr-cephadm-18.0.0-4757.g3fc0089 1.1 MB/s | 129 kB 00:00 2023-07-14T11:48:38.650 INFO:teuthology.orchestra.run.smithi097.stdout:(29/131): rbd-mirror-18.0.0-4757.g3fc0089c.el8. 6.7 MB/s | 3.4 MB 00:00 2023-07-14T11:48:38.950 INFO:teuthology.orchestra.run.smithi097.stdout:(30/131): ceph-mgr-dashboard-18.0.0-4757.g3fc00 3.5 MB/s | 1.7 MB 00:00 2023-07-14T11:48:39.101 INFO:teuthology.orchestra.run.smithi097.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4757.g3f 1.6 MB/s | 240 kB 00:00 2023-07-14T11:48:39.201 INFO:teuthology.orchestra.run.smithi097.stdout:(32/131): ceph-mgr-rook-18.0.0-4757.g3fc0089c.e 475 kB/s | 47 kB 00:00 2023-07-14T11:48:39.285 INFO:teuthology.orchestra.run.smithi097.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4757.g3 174 kB/s | 15 kB 00:00 2023-07-14T11:48:39.460 INFO:teuthology.orchestra.run.smithi097.stdout:(34/131): ceph-volume-18.0.0-4757.g3fc0089c.el8 1.4 MB/s | 258 kB 00:00 2023-07-14T11:48:39.569 INFO:teuthology.orchestra.run.smithi097.stdout:(35/131): cephadm-18.0.0-4757.g3fc0089c.el8.noa 1.5 MB/s | 166 kB 00:00 2023-07-14T11:48:39.652 INFO:teuthology.orchestra.run.smithi097.stdout:(36/131): cephfs-top-18.0.0-4757.g3fc0089c.el8. 216 kB/s | 18 kB 00:00 2023-07-14T11:48:39.869 INFO:teuthology.orchestra.run.smithi097.stdout:(37/131): ceph-mgr-diskprediction-local-18.0.0- 6.1 MB/s | 7.4 MB 00:01 2023-07-14T11:48:40.419 INFO:teuthology.orchestra.run.smithi097.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 107 kB/s | 59 kB 00:00 2023-07-14T11:48:40.611 INFO:teuthology.orchestra.run.smithi097.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 392 kB/s | 75 kB 00:00 2023-07-14T11:48:40.645 INFO:teuthology.orchestra.run.smithi097.stdout:(40/131): gperftools-libs-2.7-9.el8.x86_64.rpm 308 kB/s | 306 kB 00:00 2023-07-14T11:48:41.120 INFO:teuthology.orchestra.run.smithi097.stdout:(41/131): ceph-test-18.0.0-4757.g3fc0089c.el8.x 8.6 MB/s | 48 MB 00:05 2023-07-14T11:48:41.145 INFO:teuthology.orchestra.run.smithi097.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 87 kB/s | 44 kB 00:00 2023-07-14T11:48:41.271 INFO:teuthology.orchestra.run.smithi097.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 264 kB/s | 33 kB 00:00 2023-07-14T11:48:41.396 INFO:teuthology.orchestra.run.smithi097.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 130 kB/s | 16 kB 00:00 2023-07-14T11:48:41.430 INFO:teuthology.orchestra.run.smithi097.stdout:(45/131): python3-asyncssh-2.7.0-2.el8.noarch.r 582 kB/s | 476 kB 00:00 2023-07-14T11:48:41.571 INFO:teuthology.orchestra.run.smithi097.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 986 kB/s | 173 kB 00:00 2023-07-14T11:48:41.730 INFO:teuthology.orchestra.run.smithi097.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 513 kB/s | 81 kB 00:00 2023-07-14T11:48:41.780 INFO:teuthology.orchestra.run.smithi097.stdout:(48/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.1 MB/s | 384 kB 00:00 2023-07-14T11:48:41.822 INFO:teuthology.orchestra.run.smithi097.stdout:(49/131): python3-beautifulsoup4-4.6.3-2.el8.1. 264 kB/s | 185 kB 00:00 2023-07-14T11:48:41.856 INFO:teuthology.orchestra.run.smithi097.stdout:(50/131): python3-jaraco-6.2-6.el8.noarch.rpm 88 kB/s | 11 kB 00:00 2023-07-14T11:48:42.082 INFO:teuthology.orchestra.run.smithi097.stdout:(51/131): python3-jaraco-functools-2.0-4.el8.no 58 kB/s | 18 kB 00:00 2023-07-14T11:48:42.107 INFO:teuthology.orchestra.run.smithi097.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 194 kB/s | 49 kB 00:00 2023-07-14T11:48:42.240 INFO:teuthology.orchestra.run.smithi097.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 371 kB/s | 59 kB 00:00 2023-07-14T11:48:42.265 INFO:teuthology.orchestra.run.smithi097.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 380 kB/s | 60 kB 00:00 2023-07-14T11:48:42.399 INFO:teuthology.orchestra.run.smithi097.stdout:(55/131): python3-kubernetes-11.0.0-6.el8.noarc 2.0 MB/s | 1.2 MB 00:00 2023-07-14T11:48:42.424 INFO:teuthology.orchestra.run.smithi097.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 98 kB/s | 16 kB 00:00 2023-07-14T11:48:42.508 INFO:teuthology.orchestra.run.smithi097.stdout:(57/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.0 MB/s | 283 kB 00:00 2023-07-14T11:48:42.533 INFO:teuthology.orchestra.run.smithi097.stdout:(58/131): python3-repoze-lru-0.7-6.el8.noarch.r 251 kB/s | 34 kB 00:00 2023-07-14T11:48:42.559 INFO:teuthology.orchestra.run.smithi097.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 1.4 MB/s | 196 kB 00:00 2023-07-14T11:48:42.667 INFO:teuthology.orchestra.run.smithi097.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 140 kB/s | 19 kB 00:00 2023-07-14T11:48:42.692 INFO:teuthology.orchestra.run.smithi097.stdout:(61/131): python3-rsa-4.8-1.el8.noarch.rpm 334 kB/s | 61 kB 00:00 2023-07-14T11:48:42.718 INFO:teuthology.orchestra.run.smithi097.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 151 kB/s | 24 kB 00:00 2023-07-14T11:48:42.785 INFO:teuthology.orchestra.run.smithi097.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 250 kB/s | 29 kB 00:00 2023-07-14T11:48:42.818 INFO:teuthology.orchestra.run.smithi097.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 219 kB/s | 27 kB 00:00 2023-07-14T11:48:42.860 INFO:teuthology.orchestra.run.smithi097.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 1.8 MB/s | 255 kB 00:00 2023-07-14T11:48:42.977 INFO:teuthology.orchestra.run.smithi097.stdout:(66/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.3 MB/s | 251 kB 00:00 2023-07-14T11:48:43.002 INFO:teuthology.orchestra.run.smithi097.stdout:(67/131): python3-websocket-client-0.56.0-5.el8 333 kB/s | 61 kB 00:00 2023-07-14T11:48:43.019 INFO:teuthology.orchestra.run.smithi097.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 538 kB/s | 85 kB 00:00 2023-07-14T11:48:43.044 INFO:teuthology.orchestra.run.smithi097.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-07-14T11:48:43.103 INFO:teuthology.orchestra.run.smithi097.stdout:(70/131): python3-zc-lockfile-2.0-2.el8.noarch. 182 kB/s | 23 kB 00:00 2023-07-14T11:48:43.187 INFO:teuthology.orchestra.run.smithi097.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.3 MB/s | 110 kB 00:00 2023-07-14T11:48:43.220 INFO:teuthology.orchestra.run.smithi097.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2023-07-14T11:48:43.262 INFO:teuthology.orchestra.run.smithi097.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-07-14T11:48:43.304 INFO:teuthology.orchestra.run.smithi097.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2023-07-14T11:48:43.338 INFO:teuthology.orchestra.run.smithi097.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 1.1 MB/s | 39 kB 00:00 2023-07-14T11:48:43.405 INFO:teuthology.orchestra.run.smithi097.stdout:(76/131): bison-3.0.4-10.el8.x86_64.rpm 10 MB/s | 688 kB 00:00 2023-07-14T11:48:43.464 INFO:teuthology.orchestra.run.smithi097.stdout:(77/131): python3-jinja2-2.10.1-3.el8.noarch.rp 9.0 MB/s | 538 kB 00:00 2023-07-14T11:48:43.606 INFO:teuthology.orchestra.run.smithi097.stdout:(78/131): python3-scipy-1.0.0-21.module+el8.5.0 24 MB/s | 14 MB 00:00 2023-07-14T11:48:43.631 INFO:teuthology.orchestra.run.smithi097.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 941 kB/s | 157 kB 00:00 2023-07-14T11:48:43.656 INFO:teuthology.orchestra.run.smithi097.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2023-07-14T11:48:43.689 INFO:teuthology.orchestra.run.smithi097.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2023-07-14T11:48:43.731 INFO:teuthology.orchestra.run.smithi097.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 637 kB/s | 27 kB 00:00 2023-07-14T11:48:43.773 INFO:teuthology.orchestra.run.smithi097.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-07-14T11:48:43.815 INFO:teuthology.orchestra.run.smithi097.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 1.0 MB/s | 44 kB 00:00 2023-07-14T11:48:43.916 INFO:teuthology.orchestra.run.smithi097.stdout:(85/131): python3-babel-2.5.1-7.el8.noarch.rpm 18 MB/s | 4.8 MB 00:00 2023-07-14T11:48:43.949 INFO:teuthology.orchestra.run.smithi097.stdout:(86/131): thrift-0.13.0-2.el8.x86_64.rpm 1.8 MB/s | 1.7 MB 00:00 2023-07-14T11:48:43.983 INFO:teuthology.orchestra.run.smithi097.stdout:(87/131): python3-werkzeug-0.12.2-4.el8.noarch. 6.7 MB/s | 457 kB 00:00 2023-07-14T11:48:44.025 INFO:teuthology.orchestra.run.smithi097.stdout:(88/131): uuid-1.6.2-43.el8.x86_64.rpm 1.7 MB/s | 64 kB 00:00 2023-07-14T11:48:44.050 INFO:teuthology.orchestra.run.smithi097.stdout:(89/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 2.2 MB/s | 225 kB 00:00 2023-07-14T11:48:44.067 INFO:teuthology.orchestra.run.smithi097.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.9 MB/s | 126 kB 00:00 2023-07-14T11:48:44.093 INFO:teuthology.orchestra.run.smithi097.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 477 kB/s | 20 kB 00:00 2023-07-14T11:48:44.118 INFO:teuthology.orchestra.run.smithi097.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.0 MB/s | 103 kB 00:00 2023-07-14T11:48:44.151 INFO:teuthology.orchestra.run.smithi097.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-07-14T11:48:44.210 INFO:teuthology.orchestra.run.smithi097.stdout:(94/131): python3-numpy-1.14.3-10.el8.x86_64.rp 9.5 MB/s | 3.7 MB 00:00 2023-07-14T11:48:44.235 INFO:teuthology.orchestra.run.smithi097.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 2.3 MB/s | 198 kB 00:00 2023-07-14T11:48:44.277 INFO:teuthology.orchestra.run.smithi097.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-07-14T11:48:44.319 INFO:teuthology.orchestra.run.smithi097.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.4 MB/s | 354 kB 00:00 2023-07-14T11:48:44.403 INFO:teuthology.orchestra.run.smithi097.stdout:(98/131): openblas-threads-0.3.15-3.el8.x86_64. 17 MB/s | 4.9 MB 00:00 2023-07-14T11:48:44.478 INFO:teuthology.orchestra.run.smithi097.stdout:(99/131): openblas-0.3.15-3.el8.x86_64.rpm 18 MB/s | 4.7 MB 00:00 2023-07-14T11:48:44.520 INFO:teuthology.orchestra.run.smithi097.stdout:(100/131): postgresql-contrib-10.21-2.module+el 6.8 MB/s | 808 kB 00:00 2023-07-14T11:48:44.581 INFO:teuthology.orchestra.run.smithi097.stdout:(101/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 772 kB/s | 47 kB 00:00 2023-07-14T11:48:44.621 INFO:teuthology.orchestra.run.smithi097.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 11 MB/s | 1.5 MB 00:00 2023-07-14T11:48:44.646 INFO:teuthology.orchestra.run.smithi097.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 3.0 MB/s | 200 kB 00:00 2023-07-14T11:48:44.663 INFO:teuthology.orchestra.run.smithi097.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 1.0 MB/s | 43 kB 00:00 2023-07-14T11:48:44.688 INFO:teuthology.orchestra.run.smithi097.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 5.1 MB/s | 220 kB 00:00 2023-07-14T11:48:44.705 INFO:teuthology.orchestra.run.smithi097.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 1.1 MB/s | 48 kB 00:00 2023-07-14T11:48:44.730 INFO:teuthology.orchestra.run.smithi097.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 2.0 MB/s | 87 kB 00:00 2023-07-14T11:48:44.755 INFO:teuthology.orchestra.run.smithi097.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 851 kB/s | 43 kB 00:00 2023-07-14T11:48:44.780 INFO:teuthology.orchestra.run.smithi097.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.1 MB/s | 109 kB 00:00 2023-07-14T11:48:44.806 INFO:teuthology.orchestra.run.smithi097.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.5 MB/s | 229 kB 00:00 2023-07-14T11:48:44.873 INFO:teuthology.orchestra.run.smithi097.stdout:(111/131): postgresql-server-10.21-2.module+el8 9.1 MB/s | 5.1 MB 00:00 2023-07-14T11:48:44.898 INFO:teuthology.orchestra.run.smithi097.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 203 kB/s | 19 kB 00:00 2023-07-14T11:48:44.923 INFO:teuthology.orchestra.run.smithi097.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 3.0 MB/s | 155 kB 00:00 2023-07-14T11:48:44.957 INFO:teuthology.orchestra.run.smithi097.stdout:(114/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.1 MB/s | 560 kB 00:00 2023-07-14T11:48:44.982 INFO:teuthology.orchestra.run.smithi097.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 306 kB/s | 18 kB 00:00 2023-07-14T11:48:45.016 INFO:teuthology.orchestra.run.smithi097.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 4.7 MB/s | 559 kB 00:00 2023-07-14T11:48:45.041 INFO:teuthology.orchestra.run.smithi097.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-07-14T11:48:45.067 INFO:teuthology.orchestra.run.smithi097.stdout:(118/131): pcre2-devel-10.32-2.el8.x86_64.rpm 5.4 MB/s | 605 kB 00:00 2023-07-14T11:48:45.092 INFO:teuthology.orchestra.run.smithi097.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.4 MB/s | 72 kB 00:00 2023-07-14T11:48:45.117 INFO:teuthology.orchestra.run.smithi097.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-07-14T11:48:45.142 INFO:teuthology.orchestra.run.smithi097.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.2 MB/s | 60 kB 00:00 2023-07-14T11:48:45.162 INFO:teuthology.orchestra.run.smithi093.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-14T11:48:45.163 INFO:teuthology.orchestra.run.smithi093.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-14T11:48:45.168 INFO:teuthology.orchestra.run.smithi097.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-07-14T11:48:45.178 INFO:teuthology.orchestra.run.smithi093.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-14T11:48:45.179 INFO:teuthology.orchestra.run.smithi093.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-14T11:48:45.180 INFO:teuthology.orchestra.run.smithi093.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-14T11:48:45.181 INFO:teuthology.orchestra.run.smithi093.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-14T11:48:45.184 INFO:teuthology.orchestra.run.smithi093.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-14T11:48:45.187 INFO:teuthology.orchestra.run.smithi093.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-14T11:48:45.188 INFO:teuthology.orchestra.run.smithi093.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-14T11:48:45.193 INFO:teuthology.orchestra.run.smithi097.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-07-14T11:48:45.218 INFO:teuthology.orchestra.run.smithi097.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 768 kB/s | 39 kB 00:00 2023-07-14T11:48:45.284 INFO:teuthology.orchestra.run.smithi097.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 6.9 MB/s | 644 kB 00:00 2023-07-14T11:48:45.319 INFO:teuthology.orchestra.run.smithi097.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 1.7 MB/s | 171 kB 00:00 2023-07-14T11:48:45.320 INFO:teuthology.orchestra.run.smithi093.stdout:Dependencies resolved. 2023-07-14T11:48:45.332 INFO:teuthology.orchestra.run.smithi093.stdout:=========================================================================================================================== 2023-07-14T11:48:45.333 INFO:teuthology.orchestra.run.smithi093.stdout: Package Arch Version Repository Size 2023-07-14T11:48:45.333 INFO:teuthology.orchestra.run.smithi093.stdout:=========================================================================================================================== 2023-07-14T11:48:45.333 INFO:teuthology.orchestra.run.smithi093.stdout:Installing: 2023-07-14T11:48:45.334 INFO:teuthology.orchestra.run.smithi093.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-14T11:48:45.334 INFO:teuthology.orchestra.run.smithi093.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-14T11:48:45.334 INFO:teuthology.orchestra.run.smithi093.stdout: ceph x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 6.4 k 2023-07-14T11:48:45.334 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-base x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 5.4 M 2023-07-14T11:48:45.335 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-fuse x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 848 k 2023-07-14T11:48:45.335 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 165 k 2023-07-14T11:48:45.335 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 1.5 M 2023-07-14T11:48:45.335 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 129 k 2023-07-14T11:48:45.335 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 1.7 M 2023-07-14T11:48:45.336 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 7.4 M 2023-07-14T11:48:45.336 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-rook noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 47 k 2023-07-14T11:48:45.336 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-radosgw x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 13 M 2023-07-14T11:48:45.336 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-test x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 48 M 2023-07-14T11:48:45.337 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-volume noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 258 k 2023-07-14T11:48:45.337 INFO:teuthology.orchestra.run.smithi093.stdout: cephadm noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 166 k 2023-07-14T11:48:45.338 INFO:teuthology.orchestra.run.smithi093.stdout: cephfs-mirror x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 258 k 2023-07-14T11:48:45.338 INFO:teuthology.orchestra.run.smithi093.stdout: cephfs-top noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 18 k 2023-07-14T11:48:45.338 INFO:teuthology.orchestra.run.smithi093.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-14T11:48:45.339 INFO:teuthology.orchestra.run.smithi093.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-14T11:48:45.339 INFO:teuthology.orchestra.run.smithi093.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-14T11:48:45.340 INFO:teuthology.orchestra.run.smithi093.stdout: libcephfs-devel x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 31 k 2023-07-14T11:48:45.340 INFO:teuthology.orchestra.run.smithi093.stdout: libcephfs2 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 771 k 2023-07-14T11:48:45.340 INFO:teuthology.orchestra.run.smithi093.stdout: librados-devel x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 139 k 2023-07-14T11:48:45.341 INFO:teuthology.orchestra.run.smithi093.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-14T11:48:45.341 INFO:teuthology.orchestra.run.smithi093.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-14T11:48:45.341 INFO:teuthology.orchestra.run.smithi093.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-14T11:48:45.341 INFO:teuthology.orchestra.run.smithi093.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-14T11:48:45.342 INFO:teuthology.orchestra.run.smithi093.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-14T11:48:45.342 INFO:teuthology.orchestra.run.smithi093.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-14T11:48:45.342 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cephfs x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 197 k 2023-07-14T11:48:45.343 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rados x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 371 k 2023-07-14T11:48:45.343 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rbd x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 364 k 2023-07-14T11:48:45.343 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rgw x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 107 k 2023-07-14T11:48:45.344 INFO:teuthology.orchestra.run.smithi093.stdout: rbd-fuse x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 94 k 2023-07-14T11:48:45.344 INFO:teuthology.orchestra.run.smithi093.stdout: rbd-mirror x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 3.4 M 2023-07-14T11:48:45.344 INFO:teuthology.orchestra.run.smithi093.stdout: rbd-nbd x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 179 k 2023-07-14T11:48:45.345 INFO:teuthology.orchestra.run.smithi093.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-14T11:48:45.345 INFO:teuthology.orchestra.run.smithi093.stdout:Upgrading: 2023-07-14T11:48:45.345 INFO:teuthology.orchestra.run.smithi093.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-14T11:48:45.345 INFO:teuthology.orchestra.run.smithi093.stdout: librados2 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 3.7 M 2023-07-14T11:48:45.346 INFO:teuthology.orchestra.run.smithi093.stdout: librbd1 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 3.6 M 2023-07-14T11:48:45.346 INFO:teuthology.orchestra.run.smithi093.stdout:Installing dependencies: 2023-07-14T11:48:45.346 INFO:teuthology.orchestra.run.smithi093.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-14T11:48:45.346 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-common x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 22 M 2023-07-14T11:48:45.346 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 23 k 2023-07-14T11:48:45.347 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mds x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 2.3 M 2023-07-14T11:48:45.347 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 240 k 2023-07-14T11:48:45.347 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mon x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 4.6 M 2023-07-14T11:48:45.347 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-osd x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 18 M 2023-07-14T11:48:45.347 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 15 k 2023-07-14T11:48:45.347 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-selinux x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 24 k 2023-07-14T11:48:45.348 INFO:teuthology.orchestra.run.smithi093.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-14T11:48:45.348 INFO:teuthology.orchestra.run.smithi093.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-14T11:48:45.348 INFO:teuthology.orchestra.run.smithi093.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-14T11:48:45.348 INFO:teuthology.orchestra.run.smithi093.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-14T11:48:45.349 INFO:teuthology.orchestra.run.smithi093.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-14T11:48:45.349 INFO:teuthology.orchestra.run.smithi093.stdout: libcephsqlite x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 137 k 2023-07-14T11:48:45.349 INFO:teuthology.orchestra.run.smithi093.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-14T11:48:45.349 INFO:teuthology.orchestra.run.smithi093.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-14T11:48:45.349 INFO:teuthology.orchestra.run.smithi093.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-14T11:48:45.349 INFO:teuthology.orchestra.run.smithi093.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-14T11:48:45.350 INFO:teuthology.orchestra.run.smithi093.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-14T11:48:45.350 INFO:teuthology.orchestra.run.smithi093.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-14T11:48:45.350 INFO:teuthology.orchestra.run.smithi093.stdout: libradosstriper1 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 473 k 2023-07-14T11:48:45.350 INFO:teuthology.orchestra.run.smithi093.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-14T11:48:45.350 INFO:teuthology.orchestra.run.smithi093.stdout: librgw2 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 6.7 M 2023-07-14T11:48:45.351 INFO:teuthology.orchestra.run.smithi093.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-14T11:48:45.351 INFO:teuthology.orchestra.run.smithi093.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-14T11:48:45.351 INFO:teuthology.orchestra.run.smithi093.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-14T11:48:45.351 INFO:teuthology.orchestra.run.smithi093.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-14T11:48:45.351 INFO:teuthology.orchestra.run.smithi093.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-14T11:48:45.351 INFO:teuthology.orchestra.run.smithi093.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-14T11:48:45.352 INFO:teuthology.orchestra.run.smithi093.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-14T11:48:45.352 INFO:teuthology.orchestra.run.smithi093.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-14T11:48:45.352 INFO:teuthology.orchestra.run.smithi093.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-14T11:48:45.352 INFO:teuthology.orchestra.run.smithi093.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-14T11:48:45.353 INFO:teuthology.orchestra.run.smithi093.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-14T11:48:45.353 INFO:teuthology.orchestra.run.smithi093.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-14T11:48:45.353 INFO:teuthology.orchestra.run.smithi093.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-14T11:48:45.353 INFO:teuthology.orchestra.run.smithi093.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-14T11:48:45.353 INFO:teuthology.orchestra.run.smithi093.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-14T11:48:45.353 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-14T11:48:45.354 INFO:teuthology.orchestra.run.smithi093.stdout: python3-ceph-argparse x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 45 k 2023-07-14T11:48:45.354 INFO:teuthology.orchestra.run.smithi093.stdout: python3-ceph-common x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 119 k 2023-07-14T11:48:45.354 INFO:teuthology.orchestra.run.smithi093.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-14T11:48:45.354 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-14T11:48:45.354 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-14T11:48:45.354 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-14T11:48:45.354 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-14T11:48:45.355 INFO:teuthology.orchestra.run.smithi093.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-14T11:48:45.355 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-14T11:48:45.355 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-14T11:48:45.356 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-14T11:48:45.356 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-14T11:48:45.356 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-14T11:48:45.356 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-14T11:48:45.357 INFO:teuthology.orchestra.run.smithi093.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-14T11:48:45.357 INFO:teuthology.orchestra.run.smithi093.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-14T11:48:45.357 INFO:teuthology.orchestra.run.smithi093.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-14T11:48:45.357 INFO:teuthology.orchestra.run.smithi093.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-14T11:48:45.358 INFO:teuthology.orchestra.run.smithi093.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-14T11:48:45.358 INFO:teuthology.orchestra.run.smithi093.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-14T11:48:45.359 INFO:teuthology.orchestra.run.smithi093.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-14T11:48:45.359 INFO:teuthology.orchestra.run.smithi093.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-14T11:48:45.359 INFO:teuthology.orchestra.run.smithi093.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-14T11:48:45.359 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-14T11:48:45.360 INFO:teuthology.orchestra.run.smithi093.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-14T11:48:45.360 INFO:teuthology.orchestra.run.smithi093.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-14T11:48:45.360 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-14T11:48:45.361 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-14T11:48:45.361 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-14T11:48:45.361 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-14T11:48:45.361 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-14T11:48:45.362 INFO:teuthology.orchestra.run.smithi093.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-14T11:48:45.362 INFO:teuthology.orchestra.run.smithi093.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-14T11:48:45.362 INFO:teuthology.orchestra.run.smithi093.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-14T11:48:45.362 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-14T11:48:45.363 INFO:teuthology.orchestra.run.smithi093.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-14T11:48:45.363 INFO:teuthology.orchestra.run.smithi093.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-14T11:48:45.363 INFO:teuthology.orchestra.run.smithi093.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-14T11:48:45.363 INFO:teuthology.orchestra.run.smithi093.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-14T11:48:45.364 INFO:teuthology.orchestra.run.smithi093.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-14T11:48:45.364 INFO:teuthology.orchestra.run.smithi093.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-14T11:48:45.364 INFO:teuthology.orchestra.run.smithi093.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-14T11:48:45.364 INFO:teuthology.orchestra.run.smithi093.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-14T11:48:45.365 INFO:teuthology.orchestra.run.smithi093.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-14T11:48:45.365 INFO:teuthology.orchestra.run.smithi093.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-14T11:48:45.365 INFO:teuthology.orchestra.run.smithi093.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-14T11:48:45.365 INFO:teuthology.orchestra.run.smithi093.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-14T11:48:45.366 INFO:teuthology.orchestra.run.smithi093.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-14T11:48:45.366 INFO:teuthology.orchestra.run.smithi093.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-14T11:48:45.366 INFO:teuthology.orchestra.run.smithi093.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-14T11:48:45.367 INFO:teuthology.orchestra.run.smithi093.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-14T11:48:45.367 INFO:teuthology.orchestra.run.smithi093.stdout:Enabling module streams: 2023-07-14T11:48:45.367 INFO:teuthology.orchestra.run.smithi093.stdout: postgresql 10 2023-07-14T11:48:45.367 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-14T11:48:45.367 INFO:teuthology.orchestra.run.smithi093.stdout:Transaction Summary 2023-07-14T11:48:45.368 INFO:teuthology.orchestra.run.smithi093.stdout:=========================================================================================================================== 2023-07-14T11:48:45.368 INFO:teuthology.orchestra.run.smithi093.stdout:Install 128 Packages 2023-07-14T11:48:45.368 INFO:teuthology.orchestra.run.smithi093.stdout:Upgrade 3 Packages 2023-07-14T11:48:45.368 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-14T11:48:45.369 INFO:teuthology.orchestra.run.smithi097.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.2 MB/s | 138 kB 00:00 2023-07-14T11:48:45.369 INFO:teuthology.orchestra.run.smithi093.stdout:Total download size: 205 M 2023-07-14T11:48:45.369 INFO:teuthology.orchestra.run.smithi093.stdout:Downloading Packages: 2023-07-14T11:48:45.394 INFO:teuthology.orchestra.run.smithi097.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 6.1 MB/s | 2.3 MB 00:00 2023-07-14T11:48:45.445 INFO:teuthology.orchestra.run.smithi097.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-07-14T11:48:46.150 INFO:teuthology.orchestra.run.smithi093.stdout:(1/131): ceph-18.0.0-4757.g3fc0089c.el8.x86_64. 20 kB/s | 6.4 kB 00:00 2023-07-14T11:48:46.203 INFO:teuthology.orchestra.run.smithi097.stdout:(130/131): librbd1-18.0.0-4757.g3fc0089c.el8.x8 4.1 MB/s | 3.6 MB 00:00 2023-07-14T11:48:46.603 INFO:teuthology.orchestra.run.smithi097.stdout:(131/131): librados2-18.0.0-4757.g3fc0089c.el8. 2.9 MB/s | 3.7 MB 00:01 2023-07-14T11:48:46.607 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2023-07-14T11:48:46.608 INFO:teuthology.orchestra.run.smithi097.stdout:Total 14 MB/s | 205 MB 00:14 2023-07-14T11:48:46.934 INFO:teuthology.orchestra.run.smithi093.stdout:(2/131): ceph-fuse-18.0.0-4757.g3fc0089c.el8.x8 1.1 MB/s | 848 kB 00:00 2023-07-14T11:48:47.109 INFO:teuthology.orchestra.run.smithi093.stdout:(3/131): ceph-immutable-object-cache-18.0.0-475 947 kB/s | 165 kB 00:00 2023-07-14T11:48:47.309 INFO:teuthology.orchestra.run.smithi093.stdout:(4/131): ceph-base-18.0.0-4757.g3fc0089c.el8.x8 3.6 MB/s | 5.4 MB 00:01 2023-07-14T11:48:47.450 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2023-07-14T11:48:47.526 INFO:teuthology.orchestra.run.smithi093.stdout:(5/131): ceph-mgr-18.0.0-4757.g3fc0089c.el8.x86 7.2 MB/s | 1.5 MB 00:00 2023-07-14T11:48:47.569 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2023-07-14T11:48:47.569 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2023-07-14T11:48:47.993 INFO:teuthology.orchestra.run.smithi093.stdout:(6/131): ceph-mds-18.0.0-4757.g3fc0089c.el8.x86 2.6 MB/s | 2.3 MB 00:00 2023-07-14T11:48:48.052 INFO:teuthology.orchestra.run.smithi093.stdout:(7/131): ceph-mon-18.0.0-4757.g3fc0089c.el8.x86 8.8 MB/s | 4.6 MB 00:00 2023-07-14T11:48:48.688 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2023-07-14T11:48:48.692 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2023-07-14T11:48:48.794 INFO:teuthology.orchestra.run.smithi093.stdout:(8/131): ceph-common-18.0.0-4757.g3fc0089c.el8. 7.5 MB/s | 22 MB 00:02 2023-07-14T11:48:48.894 INFO:teuthology.orchestra.run.smithi093.stdout:(9/131): ceph-selinux-18.0.0-4757.g3fc0089c.el8 244 kB/s | 24 kB 00:00 2023-07-14T11:48:49.529 INFO:teuthology.orchestra.run.smithi017.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-14T11:48:49.530 INFO:teuthology.orchestra.run.smithi017.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-14T11:48:49.546 INFO:teuthology.orchestra.run.smithi017.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-14T11:48:49.547 INFO:teuthology.orchestra.run.smithi017.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-14T11:48:49.548 INFO:teuthology.orchestra.run.smithi017.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-14T11:48:49.549 INFO:teuthology.orchestra.run.smithi017.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-14T11:48:49.552 INFO:teuthology.orchestra.run.smithi017.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-14T11:48:49.555 INFO:teuthology.orchestra.run.smithi017.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-14T11:48:49.556 INFO:teuthology.orchestra.run.smithi017.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-14T11:48:49.704 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2023-07-14T11:48:49.718 INFO:teuthology.orchestra.run.smithi017.stdout:=========================================================================================================================== 2023-07-14T11:48:49.718 INFO:teuthology.orchestra.run.smithi017.stdout: Package Arch Version Repository Size 2023-07-14T11:48:49.718 INFO:teuthology.orchestra.run.smithi017.stdout:=========================================================================================================================== 2023-07-14T11:48:49.718 INFO:teuthology.orchestra.run.smithi017.stdout:Installing: 2023-07-14T11:48:49.718 INFO:teuthology.orchestra.run.smithi017.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-14T11:48:49.718 INFO:teuthology.orchestra.run.smithi017.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-14T11:48:49.719 INFO:teuthology.orchestra.run.smithi017.stdout: ceph x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 6.4 k 2023-07-14T11:48:49.719 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-base x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 5.4 M 2023-07-14T11:48:49.719 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-fuse x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 848 k 2023-07-14T11:48:49.719 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 165 k 2023-07-14T11:48:49.719 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 1.5 M 2023-07-14T11:48:49.719 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 129 k 2023-07-14T11:48:49.719 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 1.7 M 2023-07-14T11:48:49.719 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 7.4 M 2023-07-14T11:48:49.720 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-rook noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 47 k 2023-07-14T11:48:49.720 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-radosgw x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 13 M 2023-07-14T11:48:49.720 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-test x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 48 M 2023-07-14T11:48:49.720 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-volume noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 258 k 2023-07-14T11:48:49.720 INFO:teuthology.orchestra.run.smithi017.stdout: cephadm noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 166 k 2023-07-14T11:48:49.720 INFO:teuthology.orchestra.run.smithi017.stdout: cephfs-mirror x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 258 k 2023-07-14T11:48:49.720 INFO:teuthology.orchestra.run.smithi017.stdout: cephfs-top noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 18 k 2023-07-14T11:48:49.721 INFO:teuthology.orchestra.run.smithi017.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-14T11:48:49.721 INFO:teuthology.orchestra.run.smithi017.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-14T11:48:49.721 INFO:teuthology.orchestra.run.smithi017.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-14T11:48:49.721 INFO:teuthology.orchestra.run.smithi017.stdout: libcephfs-devel x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 31 k 2023-07-14T11:48:49.721 INFO:teuthology.orchestra.run.smithi017.stdout: libcephfs2 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 771 k 2023-07-14T11:48:49.721 INFO:teuthology.orchestra.run.smithi017.stdout: librados-devel x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 139 k 2023-07-14T11:48:49.721 INFO:teuthology.orchestra.run.smithi017.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-14T11:48:49.722 INFO:teuthology.orchestra.run.smithi017.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-14T11:48:49.722 INFO:teuthology.orchestra.run.smithi017.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-14T11:48:49.722 INFO:teuthology.orchestra.run.smithi017.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-14T11:48:49.722 INFO:teuthology.orchestra.run.smithi017.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-14T11:48:49.722 INFO:teuthology.orchestra.run.smithi017.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-14T11:48:49.722 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cephfs x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 197 k 2023-07-14T11:48:49.722 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rados x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 371 k 2023-07-14T11:48:49.722 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rbd x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 364 k 2023-07-14T11:48:49.723 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rgw x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 107 k 2023-07-14T11:48:49.723 INFO:teuthology.orchestra.run.smithi017.stdout: rbd-fuse x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 94 k 2023-07-14T11:48:49.723 INFO:teuthology.orchestra.run.smithi017.stdout: rbd-mirror x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 3.4 M 2023-07-14T11:48:49.723 INFO:teuthology.orchestra.run.smithi017.stdout: rbd-nbd x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 179 k 2023-07-14T11:48:49.723 INFO:teuthology.orchestra.run.smithi017.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-14T11:48:49.723 INFO:teuthology.orchestra.run.smithi017.stdout:Upgrading: 2023-07-14T11:48:49.723 INFO:teuthology.orchestra.run.smithi017.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-14T11:48:49.723 INFO:teuthology.orchestra.run.smithi017.stdout: librados2 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 3.7 M 2023-07-14T11:48:49.724 INFO:teuthology.orchestra.run.smithi017.stdout: librbd1 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 3.6 M 2023-07-14T11:48:49.724 INFO:teuthology.orchestra.run.smithi017.stdout:Installing dependencies: 2023-07-14T11:48:49.724 INFO:teuthology.orchestra.run.smithi017.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-14T11:48:49.724 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-common x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 22 M 2023-07-14T11:48:49.724 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 23 k 2023-07-14T11:48:49.724 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mds x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 2.3 M 2023-07-14T11:48:49.724 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 240 k 2023-07-14T11:48:49.725 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mon x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 4.6 M 2023-07-14T11:48:49.725 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-osd x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 18 M 2023-07-14T11:48:49.725 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 15 k 2023-07-14T11:48:49.725 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-selinux x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 24 k 2023-07-14T11:48:49.725 INFO:teuthology.orchestra.run.smithi017.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-14T11:48:49.725 INFO:teuthology.orchestra.run.smithi017.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-14T11:48:49.725 INFO:teuthology.orchestra.run.smithi017.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-14T11:48:49.726 INFO:teuthology.orchestra.run.smithi017.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-14T11:48:49.726 INFO:teuthology.orchestra.run.smithi017.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-14T11:48:49.726 INFO:teuthology.orchestra.run.smithi017.stdout: libcephsqlite x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 137 k 2023-07-14T11:48:49.726 INFO:teuthology.orchestra.run.smithi017.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-14T11:48:49.726 INFO:teuthology.orchestra.run.smithi017.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-14T11:48:49.726 INFO:teuthology.orchestra.run.smithi017.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-14T11:48:49.726 INFO:teuthology.orchestra.run.smithi017.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-14T11:48:49.726 INFO:teuthology.orchestra.run.smithi017.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-14T11:48:49.727 INFO:teuthology.orchestra.run.smithi017.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-14T11:48:49.727 INFO:teuthology.orchestra.run.smithi017.stdout: libradosstriper1 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 473 k 2023-07-14T11:48:49.727 INFO:teuthology.orchestra.run.smithi017.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-14T11:48:49.727 INFO:teuthology.orchestra.run.smithi017.stdout: librgw2 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 6.7 M 2023-07-14T11:48:49.727 INFO:teuthology.orchestra.run.smithi017.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-14T11:48:49.727 INFO:teuthology.orchestra.run.smithi017.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-14T11:48:49.727 INFO:teuthology.orchestra.run.smithi017.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-14T11:48:49.727 INFO:teuthology.orchestra.run.smithi017.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-14T11:48:49.728 INFO:teuthology.orchestra.run.smithi017.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-14T11:48:49.728 INFO:teuthology.orchestra.run.smithi017.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-14T11:48:49.728 INFO:teuthology.orchestra.run.smithi017.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-14T11:48:49.728 INFO:teuthology.orchestra.run.smithi017.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-14T11:48:49.728 INFO:teuthology.orchestra.run.smithi017.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-14T11:48:49.728 INFO:teuthology.orchestra.run.smithi017.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-14T11:48:49.728 INFO:teuthology.orchestra.run.smithi017.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-14T11:48:49.729 INFO:teuthology.orchestra.run.smithi017.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-14T11:48:49.729 INFO:teuthology.orchestra.run.smithi017.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-14T11:48:49.729 INFO:teuthology.orchestra.run.smithi017.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-14T11:48:49.729 INFO:teuthology.orchestra.run.smithi017.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-14T11:48:49.729 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-14T11:48:49.729 INFO:teuthology.orchestra.run.smithi017.stdout: python3-ceph-argparse x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 45 k 2023-07-14T11:48:49.729 INFO:teuthology.orchestra.run.smithi017.stdout: python3-ceph-common x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 119 k 2023-07-14T11:48:49.729 INFO:teuthology.orchestra.run.smithi017.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-14T11:48:49.730 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-14T11:48:49.730 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-14T11:48:49.730 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-14T11:48:49.730 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-14T11:48:49.730 INFO:teuthology.orchestra.run.smithi017.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-14T11:48:49.730 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-14T11:48:49.730 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-14T11:48:49.730 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-14T11:48:49.731 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-14T11:48:49.731 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-14T11:48:49.731 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-14T11:48:49.731 INFO:teuthology.orchestra.run.smithi017.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-14T11:48:49.731 INFO:teuthology.orchestra.run.smithi017.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-14T11:48:49.731 INFO:teuthology.orchestra.run.smithi017.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-14T11:48:49.731 INFO:teuthology.orchestra.run.smithi017.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-14T11:48:49.731 INFO:teuthology.orchestra.run.smithi017.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-14T11:48:49.732 INFO:teuthology.orchestra.run.smithi017.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-14T11:48:49.732 INFO:teuthology.orchestra.run.smithi017.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-14T11:48:49.732 INFO:teuthology.orchestra.run.smithi017.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-14T11:48:49.732 INFO:teuthology.orchestra.run.smithi017.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-14T11:48:49.732 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-14T11:48:49.732 INFO:teuthology.orchestra.run.smithi017.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-14T11:48:49.732 INFO:teuthology.orchestra.run.smithi017.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-14T11:48:49.733 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-14T11:48:49.733 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-14T11:48:49.733 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-14T11:48:49.733 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-14T11:48:49.733 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-14T11:48:49.733 INFO:teuthology.orchestra.run.smithi017.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-14T11:48:49.733 INFO:teuthology.orchestra.run.smithi017.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-14T11:48:49.733 INFO:teuthology.orchestra.run.smithi017.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-14T11:48:49.734 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-14T11:48:49.734 INFO:teuthology.orchestra.run.smithi017.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-14T11:48:49.734 INFO:teuthology.orchestra.run.smithi017.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-14T11:48:49.734 INFO:teuthology.orchestra.run.smithi017.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-14T11:48:49.734 INFO:teuthology.orchestra.run.smithi017.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-14T11:48:49.734 INFO:teuthology.orchestra.run.smithi017.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-14T11:48:49.734 INFO:teuthology.orchestra.run.smithi017.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-14T11:48:49.734 INFO:teuthology.orchestra.run.smithi017.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-14T11:48:49.735 INFO:teuthology.orchestra.run.smithi017.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-14T11:48:49.735 INFO:teuthology.orchestra.run.smithi017.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-14T11:48:49.735 INFO:teuthology.orchestra.run.smithi017.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-14T11:48:49.735 INFO:teuthology.orchestra.run.smithi017.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-14T11:48:49.735 INFO:teuthology.orchestra.run.smithi017.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-14T11:48:49.735 INFO:teuthology.orchestra.run.smithi017.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-14T11:48:49.735 INFO:teuthology.orchestra.run.smithi017.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-14T11:48:49.735 INFO:teuthology.orchestra.run.smithi017.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-14T11:48:49.736 INFO:teuthology.orchestra.run.smithi017.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-14T11:48:49.736 INFO:teuthology.orchestra.run.smithi017.stdout:Enabling module streams: 2023-07-14T11:48:49.736 INFO:teuthology.orchestra.run.smithi017.stdout: postgresql 10 2023-07-14T11:48:49.736 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:48:49.736 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction Summary 2023-07-14T11:48:49.736 INFO:teuthology.orchestra.run.smithi017.stdout:=========================================================================================================================== 2023-07-14T11:48:49.736 INFO:teuthology.orchestra.run.smithi017.stdout:Install 128 Packages 2023-07-14T11:48:49.737 INFO:teuthology.orchestra.run.smithi017.stdout:Upgrade 3 Packages 2023-07-14T11:48:49.737 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:48:49.737 INFO:teuthology.orchestra.run.smithi017.stdout:Total download size: 205 M 2023-07-14T11:48:49.737 INFO:teuthology.orchestra.run.smithi017.stdout:Downloading Packages: 2023-07-14T11:48:50.394 INFO:teuthology.orchestra.run.smithi017.stdout:(1/131): ceph-18.0.0-4757.g3fc0089c.el8.x86_64. 19 kB/s | 6.4 kB 00:00 2023-07-14T11:48:50.785 INFO:teuthology.orchestra.run.smithi093.stdout:(10/131): ceph-radosgw-18.0.0-4757.g3fc0089c.el 4.9 MB/s | 13 MB 00:02 2023-07-14T11:48:50.969 INFO:teuthology.orchestra.run.smithi093.stdout:(11/131): cephfs-mirror-18.0.0-4757.g3fc0089c.e 1.4 MB/s | 258 kB 00:00 2023-07-14T11:48:51.077 INFO:teuthology.orchestra.run.smithi093.stdout:(12/131): libcephfs-devel-18.0.0-4757.g3fc0089c 287 kB/s | 31 kB 00:00 2023-07-14T11:48:51.419 INFO:teuthology.orchestra.run.smithi093.stdout:(13/131): libcephfs2-18.0.0-4757.g3fc0089c.el8. 2.2 MB/s | 771 kB 00:00 2023-07-14T11:48:51.535 INFO:teuthology.orchestra.run.smithi017.stdout:(2/131): ceph-fuse-18.0.0-4757.g3fc0089c.el8.x8 743 kB/s | 848 kB 00:01 2023-07-14T11:48:51.545 INFO:teuthology.orchestra.run.smithi093.stdout:(14/131): libcephsqlite-18.0.0-4757.g3fc0089c.e 1.1 MB/s | 137 kB 00:00 2023-07-14T11:48:51.670 INFO:teuthology.orchestra.run.smithi093.stdout:(15/131): librados-devel-18.0.0-4757.g3fc0089c. 1.1 MB/s | 139 kB 00:00 2023-07-14T11:48:51.744 INFO:teuthology.orchestra.run.smithi017.stdout:(3/131): ceph-immutable-object-cache-18.0.0-475 795 kB/s | 165 kB 00:00 2023-07-14T11:48:51.895 INFO:teuthology.orchestra.run.smithi093.stdout:(16/131): libradosstriper1-18.0.0-4757.g3fc0089 2.1 MB/s | 473 kB 00:00 2023-07-14T11:48:52.529 INFO:teuthology.orchestra.run.smithi093.stdout:(17/131): ceph-test-18.0.0-4757.g3fc0089c.el8.x 13 MB/s | 48 MB 00:03 2023-07-14T11:48:52.688 INFO:teuthology.orchestra.run.smithi093.stdout:(18/131): python3-ceph-argparse-18.0.0-4757.g3f 282 kB/s | 45 kB 00:00 2023-07-14T11:48:52.838 INFO:teuthology.orchestra.run.smithi093.stdout:(19/131): python3-ceph-common-18.0.0-4757.g3fc0 792 kB/s | 119 kB 00:00 2023-07-14T11:48:52.988 INFO:teuthology.orchestra.run.smithi093.stdout:(20/131): python3-cephfs-18.0.0-4757.g3fc0089c. 1.3 MB/s | 197 kB 00:00 2023-07-14T11:48:53.139 INFO:teuthology.orchestra.run.smithi093.stdout:(21/131): python3-rados-18.0.0-4757.g3fc0089c.e 2.4 MB/s | 371 kB 00:00 2023-07-14T11:48:53.297 INFO:teuthology.orchestra.run.smithi093.stdout:(22/131): python3-rbd-18.0.0-4757.g3fc0089c.el8 2.3 MB/s | 364 kB 00:00 2023-07-14T11:48:53.472 INFO:teuthology.orchestra.run.smithi093.stdout:(23/131): python3-rgw-18.0.0-4757.g3fc0089c.el8 610 kB/s | 107 kB 00:00 2023-07-14T11:48:53.531 INFO:teuthology.orchestra.run.smithi017.stdout:(4/131): ceph-base-18.0.0-4757.g3fc0089c.el8.x8 1.6 MB/s | 5.4 MB 00:03 2023-07-14T11:48:53.639 INFO:teuthology.orchestra.run.smithi093.stdout:(24/131): rbd-fuse-18.0.0-4757.g3fc0089c.el8.x8 566 kB/s | 94 kB 00:00 2023-07-14T11:48:53.728 INFO:teuthology.orchestra.run.smithi017.stdout:(5/131): ceph-mds-18.0.0-4757.g3fc0089c.el8.x86 1.2 MB/s | 2.3 MB 00:01 2023-07-14T11:48:53.795 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2023-07-14T11:48:53.980 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-14T11:48:54.123 INFO:teuthology.orchestra.run.smithi093.stdout:(25/131): rbd-mirror-18.0.0-4757.g3fc0089c.el8. 7.1 MB/s | 3.4 MB 00:00 2023-07-14T11:48:54.128 INFO:teuthology.orchestra.run.smithi017.stdout:(6/131): ceph-mgr-18.0.0-4757.g3fc0089c.el8.x86 2.6 MB/s | 1.5 MB 00:00 2023-07-14T11:48:54.140 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-14T11:48:54.293 INFO:teuthology.orchestra.run.smithi093.stdout:(26/131): rbd-nbd-18.0.0-4757.g3fc0089c.el8.x86 1.0 MB/s | 179 kB 00:00 2023-07-14T11:48:54.356 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-07-14T11:48:54.465 INFO:teuthology.orchestra.run.smithi093.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4757.g 134 kB/s | 23 kB 00:00 2023-07-14T11:48:54.558 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-14T11:48:54.624 INFO:teuthology.orchestra.run.smithi093.stdout:(28/131): ceph-mgr-cephadm-18.0.0-4757.g3fc0089 814 kB/s | 129 kB 00:00 2023-07-14T11:48:54.671 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-14T11:48:54.891 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-14T11:48:54.941 INFO:teuthology.orchestra.run.smithi093.stdout:(29/131): ceph-mgr-dashboard-18.0.0-4757.g3fc00 5.4 MB/s | 1.7 MB 00:00 2023-07-14T11:48:55.033 INFO:teuthology.orchestra.run.smithi093.stdout:(30/131): librgw2-18.0.0-4757.g3fc0089c.el8.x86 2.1 MB/s | 6.7 MB 00:03 2023-07-14T11:48:55.293 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-14T11:48:55.316 INFO:teuthology.orchestra.run.smithi093.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4757.g3f 849 kB/s | 240 kB 00:00 2023-07-14T11:48:55.492 INFO:teuthology.orchestra.run.smithi093.stdout:(32/131): ceph-mgr-rook-18.0.0-4757.g3fc0089c.e 271 kB/s | 47 kB 00:00 2023-07-14T11:48:55.507 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 6/134 2023-07-14T11:48:55.650 INFO:teuthology.orchestra.run.smithi093.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4757.g3 92 kB/s | 15 kB 00:00 2023-07-14T11:48:55.786 INFO:teuthology.orchestra.run.smithi017.stdout:(7/131): ceph-mon-18.0.0-4757.g3fc0089c.el8.x86 2.2 MB/s | 4.6 MB 00:02 2023-07-14T11:48:55.817 INFO:teuthology.orchestra.run.smithi093.stdout:(34/131): ceph-volume-18.0.0-4757.g3fc0089c.el8 1.5 MB/s | 258 kB 00:00 2023-07-14T11:48:55.906 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 6/134 2023-07-14T11:48:55.976 INFO:teuthology.orchestra.run.smithi093.stdout:(35/131): ceph-mgr-diskprediction-local-18.0.0- 7.2 MB/s | 7.4 MB 00:01 2023-07-14T11:48:55.993 INFO:teuthology.orchestra.run.smithi093.stdout:(36/131): cephadm-18.0.0-4757.g3fc0089c.el8.noa 949 kB/s | 166 kB 00:00 2023-07-14T11:48:56.090 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 7/134 2023-07-14T11:48:56.126 INFO:teuthology.orchestra.run.smithi093.stdout:(37/131): cephfs-top-18.0.0-4757.g3fc0089c.el8. 120 kB/s | 18 kB 00:00 2023-07-14T11:48:56.204 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 7/134 2023-07-14T11:48:56.399 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 8/134 2023-07-14T11:48:56.483 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 8/134 2023-07-14T11:48:56.678 INFO:teuthology.orchestra.run.smithi017.stdout:(8/131): ceph-common-18.0.0-4757.g3fc0089c.el8. 3.3 MB/s | 22 MB 00:06 2023-07-14T11:48:56.705 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rados-2:18.0.0-4757.g3fc0089c.el8.x86_64 9/134 2023-07-14T11:48:56.776 INFO:teuthology.orchestra.run.smithi093.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 91 kB/s | 59 kB 00:00 2023-07-14T11:48:56.870 INFO:teuthology.orchestra.run.smithi017.stdout:(9/131): ceph-selinux-18.0.0-4757.g3fc0089c.el8 127 kB/s | 24 kB 00:00 2023-07-14T11:48:56.920 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-14T11:48:56.935 INFO:teuthology.orchestra.run.smithi093.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 474 kB/s | 75 kB 00:00 2023-07-14T11:48:56.969 INFO:teuthology.orchestra.run.smithi093.stdout:(40/131): gperftools-libs-2.7-9.el8.x86_64.rpm 314 kB/s | 306 kB 00:00 2023-07-14T11:48:57.120 INFO:teuthology.orchestra.run.smithi093.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 291 kB/s | 44 kB 00:00 2023-07-14T11:48:57.135 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-14T11:48:57.277 INFO:teuthology.orchestra.run.smithi093.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.1 MB/s | 185 kB 00:00 2023-07-14T11:48:57.336 INFO:teuthology.orchestra.run.smithi093.stdout:(43/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.2 MB/s | 476 kB 00:00 2023-07-14T11:48:57.379 INFO:teuthology.orchestra.run.smithi017.stdout:(10/131): ceph-osd-18.0.0-4757.g3fc0089c.el8.x8 5.5 MB/s | 18 MB 00:03 2023-07-14T11:48:57.486 INFO:teuthology.orchestra.run.smithi093.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 109 kB/s | 16 kB 00:00 2023-07-14T11:48:57.546 INFO:teuthology.orchestra.run.smithi017.stdout:(11/131): cephfs-mirror-18.0.0-4757.g3fc0089c.e 1.5 MB/s | 258 kB 00:00 2023-07-14T11:48:57.555 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-14T11:48:57.637 INFO:teuthology.orchestra.run.smithi093.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.1 MB/s | 173 kB 00:00 2023-07-14T11:48:57.713 INFO:teuthology.orchestra.run.smithi017.stdout:(12/131): libcephfs-devel-18.0.0-4757.g3fc0089c 186 kB/s | 31 kB 00:00 2023-07-14T11:48:57.754 INFO:teuthology.orchestra.run.smithi093.stdout:(46/131): python3-cachetools-3.1.1-4.el8.noarch 69 kB/s | 33 kB 00:00 2023-07-14T11:48:57.921 INFO:teuthology.orchestra.run.smithi017.stdout:(13/131): libcephfs2-18.0.0-4757.g3fc0089c.el8. 3.6 MB/s | 771 kB 00:00 2023-07-14T11:48:58.045 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-14T11:48:58.088 INFO:teuthology.orchestra.run.smithi017.stdout:(14/131): libcephsqlite-18.0.0-4757.g3fc0089c.e 822 kB/s | 137 kB 00:00 2023-07-14T11:48:58.182 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 14/134 2023-07-14T11:48:58.212 INFO:teuthology.orchestra.run.smithi093.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 177 kB/s | 81 kB 00:00 2023-07-14T11:48:58.238 INFO:teuthology.orchestra.run.smithi017.stdout:(15/131): librados-devel-18.0.0-4757.g3fc0089c. 929 kB/s | 139 kB 00:00 2023-07-14T11:48:58.254 INFO:teuthology.orchestra.run.smithi093.stdout:(48/131): python3-cherrypy-18.4.0-1.el8.noarch. 623 kB/s | 384 kB 00:00 2023-07-14T11:48:58.292 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 14/134 2023-07-14T11:48:58.346 INFO:teuthology.orchestra.run.smithi093.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 83 kB/s | 11 kB 00:00 2023-07-14T11:48:58.380 INFO:teuthology.orchestra.run.smithi093.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 140 kB/s | 18 kB 00:00 2023-07-14T11:48:58.432 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86 15/134 2023-07-14T11:48:58.506 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86 15/134 2023-07-14T11:48:58.522 INFO:teuthology.orchestra.run.smithi093.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 344 kB/s | 49 kB 00:00 2023-07-14T11:48:58.623 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-14T11:48:58.655 INFO:teuthology.orchestra.run.smithi093.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 440 kB/s | 59 kB 00:00 2023-07-14T11:48:58.772 INFO:teuthology.orchestra.run.smithi093.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 517 kB/s | 60 kB 00:00 2023-07-14T11:48:58.791 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-14T11:48:58.823 INFO:teuthology.orchestra.run.smithi017.stdout:(16/131): ceph-radosgw-18.0.0-4757.g3fc0089c.el 4.4 MB/s | 13 MB 00:03 2023-07-14T11:48:58.914 INFO:teuthology.orchestra.run.smithi093.stdout:(54/131): python3-kubernetes-11.0.0-6.el8.noarc 2.1 MB/s | 1.2 MB 00:00 2023-07-14T11:48:58.931 INFO:teuthology.orchestra.run.smithi017.stdout:(17/131): libradosstriper1-18.0.0-4757.g3fc0089 683 kB/s | 473 kB 00:00 2023-07-14T11:48:58.948 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-14T11:48:59.014 INFO:teuthology.orchestra.run.smithi093.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.1 MB/s | 283 kB 00:00 2023-07-14T11:48:59.056 INFO:teuthology.orchestra.run.smithi093.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 110 kB/s | 16 kB 00:00 2023-07-14T11:48:59.081 INFO:teuthology.orchestra.run.smithi017.stdout:(18/131): python3-ceph-argparse-18.0.0-4757.g3f 299 kB/s | 45 kB 00:00 2023-07-14T11:48:59.148 INFO:teuthology.orchestra.run.smithi093.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 252 kB/s | 34 kB 00:00 2023-07-14T11:48:59.199 INFO:teuthology.orchestra.run.smithi093.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 1.4 MB/s | 196 kB 00:00 2023-07-14T11:48:59.240 INFO:teuthology.orchestra.run.smithi017.stdout:(19/131): python3-ceph-common-18.0.0-4757.g3fc0 751 kB/s | 119 kB 00:00 2023-07-14T11:48:59.291 INFO:teuthology.orchestra.run.smithi093.stdout:(59/131): python3-rsa-4.8-1.el8.noarch.rpm 434 kB/s | 61 kB 00:00 2023-07-14T11:48:59.333 INFO:teuthology.orchestra.run.smithi093.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 140 kB/s | 19 kB 00:00 2023-07-14T11:48:59.415 INFO:teuthology.orchestra.run.smithi017.stdout:(20/131): python3-cephfs-18.0.0-4757.g3fc0089c. 1.1 MB/s | 197 kB 00:00 2023-07-14T11:48:59.424 INFO:teuthology.orchestra.run.smithi093.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 179 kB/s | 24 kB 00:00 2023-07-14T11:48:59.466 INFO:teuthology.orchestra.run.smithi093.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 219 kB/s | 29 kB 00:00 2023-07-14T11:48:59.550 INFO:teuthology.orchestra.run.smithi093.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 219 kB/s | 27 kB 00:00 2023-07-14T11:48:59.562 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-14T11:48:59.599 INFO:teuthology.orchestra.run.smithi017.stdout:(21/131): python3-rados-18.0.0-4757.g3fc0089c.e 2.0 MB/s | 371 kB 00:00 2023-07-14T11:48:59.600 INFO:teuthology.orchestra.run.smithi093.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 1.9 MB/s | 255 kB 00:00 2023-07-14T11:48:59.700 INFO:teuthology.orchestra.run.smithi093.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.6 MB/s | 251 kB 00:00 2023-07-14T11:48:59.732 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-14T11:48:59.743 INFO:teuthology.orchestra.run.smithi093.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 431 kB/s | 61 kB 00:00 2023-07-14T11:48:59.799 INFO:teuthology.orchestra.run.smithi017.stdout:(22/131): python3-rbd-18.0.0-4757.g3fc0089c.el8 1.8 MB/s | 364 kB 00:00 2023-07-14T11:48:59.843 INFO:teuthology.orchestra.run.smithi093.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 602 kB/s | 85 kB 00:00 2023-07-14T11:48:59.893 INFO:teuthology.orchestra.run.smithi093.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 153 kB/s | 23 kB 00:00 2023-07-14T11:48:59.918 INFO:teuthology.orchestra.run.smithi093.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-07-14T11:48:59.969 INFO:teuthology.orchestra.run.smithi093.stdout:(70/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 938 kB/s | 47 kB 00:00 2023-07-14T11:48:59.991 INFO:teuthology.orchestra.run.smithi017.stdout:(23/131): librgw2-18.0.0-4757.g3fc0089c.el8.x86 5.8 MB/s | 6.7 MB 00:01 2023-07-14T11:49:00.002 INFO:teuthology.orchestra.run.smithi093.stdout:(71/131): libselinux-devel-2.9-5.el8.x86_64.rpm 5.9 MB/s | 200 kB 00:00 2023-07-14T11:49:00.016 INFO:teuthology.orchestra.run.smithi017.stdout:(24/131): python3-rgw-18.0.0-4757.g3fc0089c.el8 492 kB/s | 107 kB 00:00 2023-07-14T11:49:00.036 INFO:teuthology.orchestra.run.smithi093.stdout:(72/131): python3-jwt-1.6.1-2.el8.noarch.rpm 1.3 MB/s | 43 kB 00:00 2023-07-14T11:49:00.099 INFO:teuthology.orchestra.run.smithi093.stdout:(73/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 3.4 MB/s | 220 kB 00:00 2023-07-14T11:49:00.133 INFO:teuthology.orchestra.run.smithi017.stdout:(25/131): rbd-fuse-18.0.0-4757.g3fc0089c.el8.x8 666 kB/s | 94 kB 00:00 2023-07-14T11:49:00.136 INFO:teuthology.orchestra.run.smithi093.stdout:(74/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 1.3 MB/s | 48 kB 00:00 2023-07-14T11:49:00.179 INFO:teuthology.orchestra.run.smithi093.stdout:(75/131): libsepol-devel-2.9-3.el8.x86_64.rpm 2.0 MB/s | 87 kB 00:00 2023-07-14T11:49:00.279 INFO:teuthology.orchestra.run.smithi093.stdout:(76/131): python3-requests-oauthlib-1.0.0-1.el8 427 kB/s | 43 kB 00:00 2023-07-14T11:49:00.308 INFO:teuthology.orchestra.run.smithi017.stdout:(26/131): rbd-nbd-18.0.0-4757.g3fc0089c.el8.x86 1.0 MB/s | 179 kB 00:00 2023-07-14T11:49:00.311 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-14T11:49:00.321 INFO:teuthology.orchestra.run.smithi093.stdout:(77/131): python3-pycparser-2.14-14.el8.noarch. 2.6 MB/s | 109 kB 00:00 2023-07-14T11:49:00.363 INFO:teuthology.orchestra.run.smithi093.stdout:(78/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 5.4 MB/s | 229 kB 00:00 2023-07-14T11:49:00.413 INFO:teuthology.orchestra.run.smithi093.stdout:(79/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 11 MB/s | 560 kB 00:00 2023-07-14T11:49:00.438 INFO:teuthology.orchestra.run.smithi093.stdout:(80/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 750 kB/s | 19 kB 00:00 2023-07-14T11:49:00.472 INFO:teuthology.orchestra.run.smithi093.stdout:(81/131): python3-oauthlib-2.1.0-1.el8.noarch.r 4.5 MB/s | 155 kB 00:00 2023-07-14T11:49:00.506 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-07-14T11:49:00.508 INFO:teuthology.orchestra.run.smithi017.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4757.g 115 kB/s | 23 kB 00:00 2023-07-14T11:49:00.514 INFO:teuthology.orchestra.run.smithi093.stdout:(82/131): python3-cryptography-3.2.1-5.el8.x86_ 13 MB/s | 559 kB 00:00 2023-07-14T11:49:00.547 INFO:teuthology.orchestra.run.smithi093.stdout:(83/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 538 kB/s | 18 kB 00:00 2023-07-14T11:49:00.598 INFO:teuthology.orchestra.run.smithi093.stdout:(84/131): pcre2-devel-10.32-2.el8.x86_64.rpm 12 MB/s | 605 kB 00:00 2023-07-14T11:49:00.700 INFO:teuthology.orchestra.run.smithi017.stdout:(28/131): rbd-mirror-18.0.0-4757.g3fc0089c.el8. 5.0 MB/s | 3.4 MB 00:00 2023-07-14T11:49:00.726 INFO:teuthology.orchestra.run.smithi017.stdout:(29/131): ceph-mgr-cephadm-18.0.0-4757.g3fc0089 594 kB/s | 129 kB 00:00 2023-07-14T11:49:00.773 INFO:teuthology.orchestra.run.smithi093.stdout:(85/131): ceph-osd-18.0.0-4757.g3fc0089c.el8.x8 1.4 MB/s | 18 MB 00:12 2023-07-14T11:49:00.807 INFO:teuthology.orchestra.run.smithi093.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 1.1 MB/s | 238 kB 00:00 2023-07-14T11:49:00.849 INFO:teuthology.orchestra.run.smithi093.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 2.0 MB/s | 72 kB 00:00 2023-07-14T11:49:00.899 INFO:teuthology.orchestra.run.smithi093.stdout:(88/131): thrift-0.13.0-2.el8.x86_64.rpm 1.6 MB/s | 1.7 MB 00:01 2023-07-14T11:49:00.941 INFO:teuthology.orchestra.run.smithi093.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 900 kB/s | 83 kB 00:00 2023-07-14T11:49:00.999 INFO:teuthology.orchestra.run.smithi093.stdout:(90/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 10 MB/s | 2.3 MB 00:00 2023-07-14T11:49:01.025 INFO:teuthology.orchestra.run.smithi093.stdout:(91/131): elfutils-libelf-devel-0.186-1.el8.x86 478 kB/s | 60 kB 00:00 2023-07-14T11:49:01.041 INFO:teuthology.orchestra.run.smithi093.stdout:(92/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.0 MB/s | 98 kB 00:00 2023-07-14T11:49:01.042 INFO:teuthology.orchestra.run.smithi017.stdout:(30/131): ceph-mgr-dashboard-18.0.0-4757.g3fc00 5.0 MB/s | 1.7 MB 00:00 2023-07-14T11:49:01.067 INFO:teuthology.orchestra.run.smithi093.stdout:(93/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 859 kB/s | 58 kB 00:00 2023-07-14T11:49:01.092 INFO:teuthology.orchestra.run.smithi093.stdout:(94/131): libcom_err-devel-1.45.6-4.el8.x86_64. 574 kB/s | 39 kB 00:00 2023-07-14T11:49:01.117 INFO:teuthology.orchestra.run.smithi093.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 3.3 MB/s | 171 kB 00:00 2023-07-14T11:49:01.142 INFO:teuthology.orchestra.run.smithi093.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-07-14T11:49:01.176 INFO:teuthology.orchestra.run.smithi093.stdout:(97/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 4.7 MB/s | 644 kB 00:00 2023-07-14T11:49:01.193 INFO:teuthology.orchestra.run.smithi017.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4757.g3f 1.6 MB/s | 240 kB 00:00 2023-07-14T11:49:01.201 INFO:teuthology.orchestra.run.smithi093.stdout:(98/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.8 MB/s | 110 kB 00:00 2023-07-14T11:49:01.219 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-07-14T11:49:01.227 INFO:teuthology.orchestra.run.smithi093.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.1 MB/s | 54 kB 00:00 2023-07-14T11:49:01.252 INFO:teuthology.orchestra.run.smithi093.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.0 MB/s | 259 kB 00:00 2023-07-14T11:49:01.285 INFO:teuthology.orchestra.run.smithi093.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2023-07-14T11:49:01.319 INFO:teuthology.orchestra.run.smithi093.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 580 kB/s | 39 kB 00:00 2023-07-14T11:49:01.351 INFO:teuthology.orchestra.run.smithi017.stdout:(32/131): ceph-mgr-rook-18.0.0-4757.g3fc0089c.e 300 kB/s | 47 kB 00:00 2023-07-14T11:49:01.352 INFO:teuthology.orchestra.run.smithi093.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 10 MB/s | 688 kB 00:00 2023-07-14T11:49:01.386 INFO:teuthology.orchestra.run.smithi093.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 7.9 MB/s | 538 kB 00:00 2023-07-14T11:49:01.412 INFO:teuthology.orchestra.run.smithi093.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-07-14T11:49:01.446 INFO:teuthology.orchestra.run.smithi093.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.1 MB/s | 187 kB 00:00 2023-07-14T11:49:01.479 INFO:teuthology.orchestra.run.smithi093.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 10 MB/s | 713 kB 00:00 2023-07-14T11:49:01.513 INFO:teuthology.orchestra.run.smithi093.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 797 kB/s | 27 kB 00:00 2023-07-14T11:49:01.527 INFO:teuthology.orchestra.run.smithi017.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4757.g3 83 kB/s | 15 kB 00:00 2023-07-14T11:49:01.530 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-07-14T11:49:01.555 INFO:teuthology.orchestra.run.smithi093.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-07-14T11:49:01.588 INFO:teuthology.orchestra.run.smithi093.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.3 MB/s | 44 kB 00:00 2023-07-14T11:49:01.669 INFO:teuthology.orchestra.run.smithi017.stdout:(34/131): ceph-volume-18.0.0-4757.g3fc0089c.el8 1.8 MB/s | 258 kB 00:00 2023-07-14T11:49:01.672 INFO:teuthology.orchestra.run.smithi093.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 21 MB/s | 4.8 MB 00:00 2023-07-14T11:49:01.714 INFO:teuthology.orchestra.run.smithi093.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 11 MB/s | 457 kB 00:00 2023-07-14T11:49:01.742 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-07-14T11:49:01.772 INFO:teuthology.orchestra.run.smithi093.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 3.8 MB/s | 225 kB 00:00 2023-07-14T11:49:01.831 INFO:teuthology.orchestra.run.smithi093.stdout:(114/131): python3-numpy-1.14.3-10.el8.x86_64.r 15 MB/s | 3.7 MB 00:00 2023-07-14T11:49:01.848 INFO:teuthology.orchestra.run.smithi093.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 852 kB/s | 64 kB 00:00 2023-07-14T11:49:01.873 INFO:teuthology.orchestra.run.smithi093.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 3.0 MB/s | 126 kB 00:00 2023-07-14T11:49:01.890 INFO:teuthology.orchestra.run.smithi093.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 478 kB/s | 20 kB 00:00 2023-07-14T11:49:01.916 INFO:teuthology.orchestra.run.smithi093.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.4 MB/s | 103 kB 00:00 2023-07-14T11:49:01.949 INFO:teuthology.orchestra.run.smithi093.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-07-14T11:49:01.985 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-07-14T11:49:01.991 INFO:teuthology.orchestra.run.smithi093.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-07-14T11:49:02.125 INFO:teuthology.orchestra.run.smithi093.stdout:(121/131): python3-scipy-1.0.0-21.module+el8.5. 14 MB/s | 14 MB 00:01 2023-07-14T11:49:02.167 INFO:teuthology.orchestra.run.smithi093.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-07-14T11:49:02.186 INFO:teuthology.orchestra.run.smithi017.stdout:(35/131): ceph-test-18.0.0-4757.g3fc0089c.el8.x 9.0 MB/s | 48 MB 00:05 2023-07-14T11:49:02.196 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-07-14T11:49:02.208 INFO:teuthology.orchestra.run.smithi093.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-07-14T11:49:02.211 INFO:teuthology.orchestra.run.smithi017.stdout:(36/131): cephadm-18.0.0-4757.g3fc0089c.el8.noa 307 kB/s | 166 kB 00:00 2023-07-14T11:49:02.300 INFO:teuthology.orchestra.run.smithi093.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 13 MB/s | 4.9 MB 00:00 2023-07-14T11:49:02.311 INFO:teuthology.orchestra.run.smithi017.stdout:(37/131): ceph-mgr-diskprediction-local-18.0.0- 4.7 MB/s | 7.4 MB 00:01 2023-07-14T11:49:02.336 INFO:teuthology.orchestra.run.smithi017.stdout:(38/131): cephfs-top-18.0.0-4757.g3fc0089c.el8. 120 kB/s | 18 kB 00:00 2023-07-14T11:49:02.384 INFO:teuthology.orchestra.run.smithi093.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 12 MB/s | 4.7 MB 00:00 2023-07-14T11:49:02.412 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-07-14T11:49:02.418 INFO:teuthology.orchestra.run.smithi093.stdout:(126/131): postgresql-contrib-10.21-2.module+el 6.8 MB/s | 808 kB 00:00 2023-07-14T11:49:02.476 INFO:teuthology.orchestra.run.smithi093.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 16 MB/s | 1.5 MB 00:00 2023-07-14T11:49:02.560 INFO:teuthology.orchestra.run.smithi093.stdout:(128/131): postgresql-server-10.21-2.module+el8 14 MB/s | 5.1 MB 00:00 2023-07-14T11:49:02.618 INFO:teuthology.orchestra.run.smithi093.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-07-14T11:49:02.650 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-14T11:49:02.829 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ceph-common-2:18.0.0-4757.g3fc0089c.el8. 26/134 2023-07-14T11:49:02.837 INFO:teuthology.orchestra.run.smithi017.stdout:(39/131): liboath-2.6.2-3.el8.x86_64.rpm 112 kB/s | 59 kB 00:00 2023-07-14T11:49:02.862 INFO:teuthology.orchestra.run.smithi017.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 143 kB/s | 75 kB 00:00 2023-07-14T11:49:02.993 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ceph-argparse-2:18.0.0-4757.g3fc0089c.el 27/134 2023-07-14T11:49:03.029 INFO:teuthology.orchestra.run.smithi017.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 261 kB/s | 44 kB 00:00 2023-07-14T11:49:03.296 INFO:teuthology.orchestra.run.smithi017.stdout:(42/131): gperftools-libs-2.7-9.el8.x86_64.rpm 282 kB/s | 306 kB 00:01 2023-07-14T11:49:03.480 INFO:teuthology.orchestra.run.smithi017.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 180 kB/s | 33 kB 00:00 2023-07-14T11:49:03.638 INFO:teuthology.orchestra.run.smithi017.stdout:(44/131): python3-beautifulsoup4-4.6.3-2.el8.1. 304 kB/s | 185 kB 00:00 2023-07-14T11:49:03.664 INFO:teuthology.orchestra.run.smithi017.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 89 kB/s | 16 kB 00:00 2023-07-14T11:49:03.901 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cephfs-2:18.0.0-4757.g3fc0089c.el8.x86_6 28/134 2023-07-14T11:49:03.972 INFO:teuthology.orchestra.run.smithi017.stdout:(46/131): python3-asyncssh-2.7.0-2.el8.noarch.r 419 kB/s | 476 kB 00:01 2023-07-14T11:49:04.039 INFO:teuthology.orchestra.run.smithi017.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 431 kB/s | 173 kB 00:00 2023-07-14T11:49:04.052 INFO:teuthology.orchestra.run.smithi093.stdout:(130/131): librados2-18.0.0-4757.g3fc0089c.el8. 2.2 MB/s | 3.7 MB 00:01 2023-07-14T11:49:04.214 INFO:teuthology.orchestra.run.smithi017.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 63 kB/s | 11 kB 00:00 2023-07-14T11:49:04.215 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-07-14T11:49:04.256 INFO:teuthology.orchestra.run.smithi017.stdout:(49/131): python3-google-auth-1.1.1-10.el8.noar 286 kB/s | 81 kB 00:00 2023-07-14T11:49:04.357 INFO:teuthology.orchestra.run.smithi017.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 123 kB/s | 18 kB 00:00 2023-07-14T11:49:04.449 INFO:teuthology.orchestra.run.smithi017.stdout:(51/131): python3-cherrypy-18.4.0-1.el8.noarch. 490 kB/s | 384 kB 00:00 2023-07-14T11:49:04.516 INFO:teuthology.orchestra.run.smithi017.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 307 kB/s | 49 kB 00:00 2023-07-14T11:49:04.517 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-07-14T11:49:04.618 INFO:teuthology.orchestra.run.smithi093.stdout:(131/131): librbd1-18.0.0-4757.g3fc0089c.el8.x8 1.7 MB/s | 3.6 MB 00:02 2023-07-14T11:49:04.622 INFO:teuthology.orchestra.run.smithi093.stdout:-------------------------------------------------------------------------------- 2023-07-14T11:49:04.623 INFO:teuthology.orchestra.run.smithi093.stdout:Total 11 MB/s | 205 MB 00:19 2023-07-14T11:49:04.641 INFO:teuthology.orchestra.run.smithi017.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 306 kB/s | 59 kB 00:00 2023-07-14T11:49:04.682 INFO:teuthology.orchestra.run.smithi017.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 361 kB/s | 60 kB 00:00 2023-07-14T11:49:04.788 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-07-14T11:49:04.841 INFO:teuthology.orchestra.run.smithi017.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 98 kB/s | 16 kB 00:00 2023-07-14T11:49:04.991 INFO:teuthology.orchestra.run.smithi017.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 224 kB/s | 34 kB 00:00 2023-07-14T11:49:05.031 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-14T11:49:05.233 INFO:teuthology.orchestra.run.smithi017.stdout:(57/131): python3-pecan-1.3.2-9.el8.noarch.rpm 477 kB/s | 283 kB 00:00 2023-07-14T11:49:05.400 INFO:teuthology.orchestra.run.smithi017.stdout:(58/131): python3-rsa-4.8-1.el8.noarch.rpm 369 kB/s | 61 kB 00:00 2023-07-14T11:49:05.438 INFO:teuthology.orchestra.run.smithi093.stdout:Running transaction check 2023-07-14T11:49:05.459 INFO:teuthology.orchestra.run.smithi017.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 420 kB/s | 196 kB 00:00 2023-07-14T11:49:05.543 INFO:teuthology.orchestra.run.smithi017.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 132 kB/s | 19 kB 00:00 2023-07-14T11:49:05.558 INFO:teuthology.orchestra.run.smithi093.stdout:Transaction check succeeded. 2023-07-14T11:49:05.559 INFO:teuthology.orchestra.run.smithi093.stdout:Running transaction test 2023-07-14T11:49:05.601 INFO:teuthology.orchestra.run.smithi017.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 169 kB/s | 24 kB 00:00 2023-07-14T11:49:05.701 INFO:teuthology.orchestra.run.smithi017.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 185 kB/s | 29 kB 00:00 2023-07-14T11:49:05.760 INFO:teuthology.orchestra.run.smithi017.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 174 kB/s | 27 kB 00:00 2023-07-14T11:49:05.960 INFO:teuthology.orchestra.run.smithi017.stdout:(64/131): python3-kubernetes-11.0.0-6.el8.noarc 701 kB/s | 1.2 MB 00:01 2023-07-14T11:49:06.127 INFO:teuthology.orchestra.run.smithi017.stdout:(65/131): python3-websocket-client-0.56.0-5.el8 367 kB/s | 61 kB 00:00 2023-07-14T11:49:06.152 INFO:teuthology.orchestra.run.smithi017.stdout:(66/131): python3-waitress-1.4.3-1.el8.noarch.r 566 kB/s | 255 kB 00:00 2023-07-14T11:49:06.178 INFO:teuthology.orchestra.run.smithi017.stdout:(67/131): python3-webob-1.8.5-1.el8.1.noarch.rp 601 kB/s | 251 kB 00:00 2023-07-14T11:49:06.287 INFO:teuthology.orchestra.run.smithi017.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 537 kB/s | 85 kB 00:00 2023-07-14T11:49:06.312 INFO:teuthology.orchestra.run.smithi017.stdout:(69/131): python3-zc-lockfile-2.0-2.el8.noarch. 144 kB/s | 23 kB 00:00 2023-07-14T11:49:06.338 INFO:teuthology.orchestra.run.smithi017.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2023-07-14T11:49:06.396 INFO:teuthology.orchestra.run.smithi017.stdout:(71/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 562 kB/s | 47 kB 00:00 2023-07-14T11:49:06.429 INFO:teuthology.orchestra.run.smithi017.stdout:(72/131): libselinux-devel-2.9-5.el8.x86_64.rpm 2.4 MB/s | 200 kB 00:00 2023-07-14T11:49:06.455 INFO:teuthology.orchestra.run.smithi017.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 730 kB/s | 43 kB 00:00 2023-07-14T11:49:06.488 INFO:teuthology.orchestra.run.smithi017.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 3.7 MB/s | 220 kB 00:00 2023-07-14T11:49:06.514 INFO:teuthology.orchestra.run.smithi017.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 821 kB/s | 48 kB 00:00 2023-07-14T11:49:06.539 INFO:teuthology.orchestra.run.smithi017.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-07-14T11:49:06.564 INFO:teuthology.orchestra.run.smithi017.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 854 kB/s | 43 kB 00:00 2023-07-14T11:49:06.590 INFO:teuthology.orchestra.run.smithi017.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 2.1 MB/s | 109 kB 00:00 2023-07-14T11:49:06.623 INFO:teuthology.orchestra.run.smithi017.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.8 MB/s | 229 kB 00:00 2023-07-14T11:49:06.681 INFO:teuthology.orchestra.run.smithi093.stdout:Transaction test succeeded. 2023-07-14T11:49:06.685 INFO:teuthology.orchestra.run.smithi093.stdout:Running transaction 2023-07-14T11:49:06.690 INFO:teuthology.orchestra.run.smithi017.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 5.5 MB/s | 560 kB 00:00 2023-07-14T11:49:06.715 INFO:teuthology.orchestra.run.smithi017.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 203 kB/s | 19 kB 00:00 2023-07-14T11:49:06.740 INFO:teuthology.orchestra.run.smithi017.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.0 MB/s | 155 kB 00:00 2023-07-14T11:49:06.783 INFO:teuthology.orchestra.run.smithi017.stdout:(83/131): python3-cryptography-3.2.1-5.el8.x86_ 8.2 MB/s | 559 kB 00:00 2023-07-14T11:49:06.808 INFO:teuthology.orchestra.run.smithi017.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 267 kB/s | 18 kB 00:00 2023-07-14T11:49:06.810 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-14T11:49:06.850 INFO:teuthology.orchestra.run.smithi017.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 8.9 MB/s | 605 kB 00:00 2023-07-14T11:49:06.883 INFO:teuthology.orchestra.run.smithi017.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.1 MB/s | 238 kB 00:00 2023-07-14T11:49:06.925 INFO:teuthology.orchestra.run.smithi017.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.7 MB/s | 72 kB 00:00 2023-07-14T11:49:06.967 INFO:teuthology.orchestra.run.smithi017.stdout:(88/131): fuse-2.9.7-15.el8.x86_64.rpm 1.9 MB/s | 83 kB 00:00 2023-07-14T11:49:07.026 INFO:teuthology.orchestra.run.smithi017.stdout:(89/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 13 MB/s | 2.3 MB 00:00 2023-07-14T11:49:07.080 INFO:teuthology.orchestra.run.smithi017.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 531 kB/s | 60 kB 00:00 2023-07-14T11:49:07.118 INFO:teuthology.orchestra.run.smithi017.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.0 MB/s | 98 kB 00:00 2023-07-14T11:49:07.143 INFO:teuthology.orchestra.run.smithi017.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 919 kB/s | 58 kB 00:00 2023-07-14T11:49:07.166 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-07-14T11:49:07.169 INFO:teuthology.orchestra.run.smithi017.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 770 kB/s | 39 kB 00:00 2023-07-14T11:49:07.211 INFO:teuthology.orchestra.run.smithi017.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 9.4 MB/s | 644 kB 00:00 2023-07-14T11:49:07.236 INFO:teuthology.orchestra.run.smithi017.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.5 MB/s | 171 kB 00:00 2023-07-14T11:49:07.261 INFO:teuthology.orchestra.run.smithi017.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-07-14T11:49:07.303 INFO:teuthology.orchestra.run.smithi017.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-07-14T11:49:07.345 INFO:teuthology.orchestra.run.smithi017.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-07-14T11:49:07.395 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-14T11:49:07.396 INFO:teuthology.orchestra.run.smithi017.stdout:(99/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.1 MB/s | 259 kB 00:00 2023-07-14T11:49:07.446 INFO:teuthology.orchestra.run.smithi017.stdout:(100/131): socat-1.7.4.1-1.el8.x86_64.rpm 6.3 MB/s | 323 kB 00:00 2023-07-14T11:49:07.513 INFO:teuthology.orchestra.run.smithi017.stdout:(101/131): python3-markupsafe-0.23-19.el8.x86_6 584 kB/s | 39 kB 00:00 2023-07-14T11:49:07.580 INFO:teuthology.orchestra.run.smithi017.stdout:(102/131): bison-3.0.4-10.el8.x86_64.rpm 10 MB/s | 688 kB 00:00 2023-07-14T11:49:07.583 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-14T11:49:07.630 INFO:teuthology.orchestra.run.smithi017.stdout:(103/131): python3-jinja2-2.10.1-3.el8.noarch.r 11 MB/s | 538 kB 00:00 2023-07-14T11:49:07.672 INFO:teuthology.orchestra.run.smithi017.stdout:(104/131): python3-mako-1.0.6-13.el8.noarch.rpm 3.7 MB/s | 157 kB 00:00 2023-07-14T11:49:07.722 INFO:teuthology.orchestra.run.smithi017.stdout:(105/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2023-07-14T11:49:07.770 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 36/134 2023-07-14T11:49:07.789 INFO:teuthology.orchestra.run.smithi017.stdout:(106/131): automake-1.16.1-7.el8.noarch.rpm 10 MB/s | 713 kB 00:00 2023-07-14T11:49:07.981 INFO:teuthology.orchestra.run.smithi017.stdout:(107/131): python3-scipy-1.0.0-21.module+el8.5. 18 MB/s | 14 MB 00:00 2023-07-14T11:49:07.985 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librados-devel-2:18.0.0-4757.g3fc0089c.el8.x86_6 37/134 2023-07-14T11:49:08.023 INFO:teuthology.orchestra.run.smithi017.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 638 kB/s | 27 kB 00:00 2023-07-14T11:49:08.073 INFO:teuthology.orchestra.run.smithi017.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 6.3 MB/s | 320 kB 00:00 2023-07-14T11:49:08.115 INFO:teuthology.orchestra.run.smithi017.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.0 MB/s | 44 kB 00:00 2023-07-14T11:49:08.158 INFO:teuthology.orchestra.run.smithi017.stdout:(111/131): thrift-0.13.0-2.el8.x86_64.rpm 897 kB/s | 1.7 MB 00:01 2023-07-14T11:49:08.163 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-07-14T11:49:08.258 INFO:teuthology.orchestra.run.smithi017.stdout:(112/131): python3-babel-2.5.1-7.el8.noarch.rpm 10 MB/s | 4.8 MB 00:00 2023-07-14T11:49:08.300 INFO:teuthology.orchestra.run.smithi017.stdout:(113/131): python3-werkzeug-0.12.2-4.el8.noarch 3.2 MB/s | 457 kB 00:00 2023-07-14T11:49:08.333 INFO:teuthology.orchestra.run.smithi017.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 2.9 MB/s | 225 kB 00:00 2023-07-14T11:49:08.358 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-07-14T11:49:08.382 INFO:teuthology.orchestra.run.smithi017.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 779 kB/s | 64 kB 00:00 2023-07-14T11:49:08.417 INFO:teuthology.orchestra.run.smithi017.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 1.5 MB/s | 126 kB 00:00 2023-07-14T11:49:08.442 INFO:teuthology.orchestra.run.smithi017.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 334 kB/s | 20 kB 00:00 2023-07-14T11:49:08.468 INFO:teuthology.orchestra.run.smithi017.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-07-14T11:49:08.510 INFO:teuthology.orchestra.run.smithi017.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 10 MB/s | 709 kB 00:00 2023-07-14T11:49:08.552 INFO:teuthology.orchestra.run.smithi017.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-07-14T11:49:08.617 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-07-14T11:49:08.618 INFO:teuthology.orchestra.run.smithi017.stdout:(121/131): python3-numpy-1.14.3-10.el8.x86_64.r 7.4 MB/s | 3.7 MB 00:00 2023-07-14T11:49:08.669 INFO:teuthology.orchestra.run.smithi017.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.0 MB/s | 202 kB 00:00 2023-07-14T11:49:08.719 INFO:teuthology.orchestra.run.smithi017.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 6.9 MB/s | 354 kB 00:00 2023-07-14T11:49:08.793 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-07-14T11:49:08.844 INFO:teuthology.orchestra.run.smithi017.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 13 MB/s | 4.9 MB 00:00 2023-07-14T11:49:08.936 INFO:teuthology.orchestra.run.smithi017.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 12 MB/s | 4.7 MB 00:00 2023-07-14T11:49:08.987 INFO:teuthology.orchestra.run.smithi017.stdout:(126/131): postgresql-contrib-10.21-2.module+el 5.6 MB/s | 808 kB 00:00 2023-07-14T11:49:09.013 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-07-14T11:49:09.054 INFO:teuthology.orchestra.run.smithi017.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 13 MB/s | 1.5 MB 00:00 2023-07-14T11:49:09.138 INFO:teuthology.orchestra.run.smithi017.stdout:(128/131): postgresql-server-10.21-2.module+el8 12 MB/s | 5.1 MB 00:00 2023-07-14T11:49:09.220 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-07-14T11:49:09.539 INFO:teuthology.orchestra.run.smithi017.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 1.9 MB/s | 798 kB 00:00 2023-07-14T11:49:09.620 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-07-14T11:49:09.911 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-07-14T11:49:10.124 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-07-14T11:49:10.130 INFO:teuthology.orchestra.run.smithi017.stdout:(130/131): librbd1-18.0.0-4757.g3fc0089c.el8.x8 3.3 MB/s | 3.6 MB 00:01 2023-07-14T11:49:10.188 INFO:teuthology.orchestra.run.smithi017.stdout:(131/131): librados2-18.0.0-4757.g3fc0089c.el8. 3.1 MB/s | 3.7 MB 00:01 2023-07-14T11:49:10.191 INFO:teuthology.orchestra.run.smithi017.stdout:-------------------------------------------------------------------------------- 2023-07-14T11:49:10.191 INFO:teuthology.orchestra.run.smithi017.stdout:Total 10 MB/s | 205 MB 00:20 2023-07-14T11:49:10.335 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-07-14T11:49:10.593 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-07-14T11:49:10.940 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-07-14T11:49:11.172 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-07-14T11:49:11.393 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-07-14T11:49:11.511 INFO:teuthology.orchestra.run.smithi093.stdout: Preparing : 1/1 2023-07-14T11:49:11.618 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-07-14T11:49:11.668 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-14T11:49:11.801 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-07-14T11:49:11.877 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction check 2023-07-14T11:49:11.885 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-14T11:49:11.950 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-07-14T11:49:12.023 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction check succeeded. 2023-07-14T11:49:12.024 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction test 2023-07-14T11:49:12.037 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-14T11:49:12.121 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-14T11:49:12.281 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-07-14T11:49:12.329 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-14T11:49:12.544 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-07-14T11:49:12.712 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-14T11:49:12.810 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-07-14T11:49:12.920 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 5/134 2023-07-14T11:49:13.037 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-07-14T11:49:13.203 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-07-14T11:49:13.299 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 5/134 2023-07-14T11:49:13.356 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-07-14T11:49:13.504 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 6/134 2023-07-14T11:49:13.555 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-07-14T11:49:13.629 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction test succeeded. 2023-07-14T11:49:13.633 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-07-14T11:49:13.634 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 6/134 2023-07-14T11:49:13.634 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction 2023-07-14T11:49:13.820 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 7/134 2023-07-14T11:49:13.884 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-07-14T11:49:13.914 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 7/134 2023-07-14T11:49:14.075 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-rados-2:18.0.0-4757.g3fc0089c.el8.x86_64 8/134 2023-07-14T11:49:14.083 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-07-14T11:49:14.241 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-07-14T11:49:14.275 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-14T11:49:14.334 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-14T11:49:14.465 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-14T11:49:14.559 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-07-14T11:49:14.640 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-14T11:49:14.739 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-07-14T11:49:14.946 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-14T11:49:15.022 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-14T11:49:15.132 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-14T11:49:15.282 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-14T11:49:15.398 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-14T11:49:15.428 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 14/134 2023-07-14T11:49:15.519 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 14/134 2023-07-14T11:49:15.628 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-14T11:49:15.670 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86 15/134 2023-07-14T11:49:15.775 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-14T11:49:15.820 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86 15/134 2023-07-14T11:49:15.923 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-14T11:49:16.065 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-07-14T11:49:16.129 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-14T11:49:16.226 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-07-14T11:49:16.373 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-14T11:49:16.401 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-07-14T11:49:16.536 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-14T11:49:16.567 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-14T11:49:16.749 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-14T11:49:16.763 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-14T11:49:16.885 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-14T11:49:17.078 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-14T11:49:17.151 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-14T11:49:17.384 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-14T11:49:17.569 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-14T11:49:17.693 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-14T11:49:17.819 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-14T11:49:17.837 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-14T11:49:18.020 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-14T11:49:18.195 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-14T11:49:18.432 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-14T11:49:18.456 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-14T11:49:18.611 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-07-14T11:49:19.053 INFO:teuthology.orchestra.run.smithi017.stdout: Preparing : 1/1 2023-07-14T11:49:19.281 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-14T11:49:19.319 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-07-14T11:49:19.547 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-14T11:49:19.551 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-07-14T11:49:19.560 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-14T11:49:19.724 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-14T11:49:19.733 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-07-14T11:49:19.854 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-14T11:49:19.883 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-14T11:49:19.962 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-14T11:49:20.099 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-14T11:49:20.137 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-14T11:49:20.150 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-ceph-common-2:18.0.0-4757.g3fc0089c.el8. 26/134 2023-07-14T11:49:20.272 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-ceph-argparse-2:18.0.0-4757.g3fc0089c.el 27/134 2023-07-14T11:49:20.448 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-14T11:49:20.474 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-cephfs-2:18.0.0-4757.g3fc0089c.el8.x86_6 28/134 2023-07-14T11:49:20.594 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-14T11:49:20.652 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-14T11:49:20.760 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-14T11:49:20.874 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 5/134 2023-07-14T11:49:20.975 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-14T11:49:21.355 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 5/134 2023-07-14T11:49:21.544 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 6/134 2023-07-14T11:49:21.690 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 6/134 2023-07-14T11:49:21.802 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-14T11:49:21.891 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 7/134 2023-07-14T11:49:22.012 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 92/134 2023-07-14T11:49:22.050 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 7/134 2023-07-14T11:49:22.070 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 92/134 2023-07-14T11:49:22.230 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-rados-2:18.0.0-4757.g3fc0089c.el8.x86_64 8/134 2023-07-14T11:49:22.262 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 93/134 2023-07-14T11:49:22.412 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-14T11:49:22.454 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-07-14T11:49:22.728 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-14T11:49:22.980 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-14T11:49:23.416 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-14T11:49:23.440 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-07-14T11:49:23.662 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-14T11:49:23.749 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-07-14T11:49:23.840 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 14/134 2023-07-14T11:49:23.973 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 14/134 2023-07-14T11:49:24.004 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-07-14T11:49:24.209 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-07-14T11:49:24.307 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86 15/134 2023-07-14T11:49:24.438 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-14T11:49:24.483 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86 15/134 2023-07-14T11:49:24.620 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-14T11:49:24.731 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-07-14T11:49:24.797 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-rbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 36/134 2023-07-14T11:49:24.958 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : librados-devel-2:18.0.0-4757.g3fc0089c.el8.x86_6 37/134 2023-07-14T11:49:24.967 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-07-14T11:49:25.134 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-07-14T11:49:25.171 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-07-14T11:49:25.269 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-07-14T11:49:25.340 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-14T11:49:25.435 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-07-14T11:49:25.500 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-14T11:49:25.534 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-07-14T11:49:25.681 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-14T11:49:25.757 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-07-14T11:49:25.898 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 94/134 2023-07-14T11:49:26.005 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-07-14T11:49:26.152 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-14T11:49:26.162 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 94/134 2023-07-14T11:49:26.229 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-14T11:49:26.434 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-14T11:49:26.447 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-07-14T11:49:26.597 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-07-14T11:49:26.673 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-14T11:49:26.747 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 94/134 2023-07-14T11:49:26.784 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-07-14T11:49:26.960 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-07-14T11:49:27.073 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 95/134 2023-07-14T11:49:27.129 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-07-14T11:49:27.139 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 95/134 2023-07-14T11:49:27.269 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 96/134 2023-07-14T11:49:27.289 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-07-14T11:49:27.398 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-14T11:49:27.475 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-07-14T11:49:27.598 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-07-14T11:49:27.698 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-07-14T11:49:28.057 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-07-14T11:49:28.328 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-07-14T11:49:28.481 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-07-14T11:49:28.571 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-07-14T11:49:28.764 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-07-14T11:49:28.780 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-07-14T11:49:28.972 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-07-14T11:49:28.998 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-07-14T11:49:29.255 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-14T11:49:29.339 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-07-14T11:49:29.472 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-ceph-common-2:18.0.0-4757.g3fc0089c.el8. 26/134 2023-07-14T11:49:29.525 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-07-14T11:49:29.621 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-ceph-argparse-2:18.0.0-4757.g3fc0089c.el 27/134 2023-07-14T11:49:29.690 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-07-14T11:49:29.914 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-07-14T11:49:30.105 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-07-14T11:49:30.119 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-cephfs-2:18.0.0-4757.g3fc0089c.el8.x86_6 28/134 2023-07-14T11:49:30.271 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-07-14T11:49:30.356 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-14T11:49:30.568 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-07-14T11:49:30.832 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-07-14T11:49:31.106 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-07-14T11:49:31.352 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-07-14T11:49:31.550 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-14T11:49:31.762 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-14T11:49:32.027 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-14T11:49:32.240 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-14T11:49:32.388 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-14T11:49:32.547 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-14T11:49:32.570 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-14T11:49:32.750 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-14T11:49:32.937 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-14T11:49:33.093 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-14T11:49:33.254 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-14T11:49:33.423 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-14T11:49:33.662 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-14T11:49:33.762 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-07-14T11:49:33.879 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-14T11:49:34.072 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-14T11:49:34.154 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-07-14T11:49:34.355 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-14T11:49:34.469 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-07-14T11:49:34.566 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-14T11:49:34.755 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-14T11:49:34.874 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-07-14T11:49:34.950 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-14T11:49:35.108 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-14T11:49:35.356 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-14T11:49:35.565 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-rbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 36/134 2023-07-14T11:49:35.807 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : librados-devel-2:18.0.0-4757.g3fc0089c.el8.x86_6 37/134 2023-07-14T11:49:35.975 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-07-14T11:49:36.063 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-14T11:49:36.178 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-07-14T11:49:36.320 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-14T11:49:36.355 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-07-14T11:49:36.444 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-07-14T11:49:36.559 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-14T11:49:36.751 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-07-14T11:49:36.904 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-14T11:49:37.071 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-07-14T11:49:37.185 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-14T11:49:37.238 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-14T11:49:37.365 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-14T11:49:37.372 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-14T11:49:37.624 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-07-14T11:49:37.844 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-07-14T11:49:38.025 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-07-14T11:49:38.132 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-14T11:49:38.261 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-07-14T11:49:38.300 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 92/134 2023-07-14T11:49:38.375 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 92/134 2023-07-14T11:49:38.475 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-07-14T11:49:38.583 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-rgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 93/134 2023-07-14T11:49:38.688 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-07-14T11:49:38.940 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-07-14T11:49:39.216 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-07-14T11:49:39.710 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-07-14T11:49:40.080 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-07-14T11:49:40.340 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-07-14T11:49:40.599 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-07-14T11:49:40.851 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-07-14T11:49:41.263 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-07-14T11:49:41.515 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-07-14T11:49:41.750 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-07-14T11:49:42.027 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-07-14T11:49:42.201 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 94/134 2023-07-14T11:49:42.230 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-07-14T11:49:42.408 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 94/134 2023-07-14T11:49:42.428 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-07-14T11:49:42.792 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-07-14T11:49:43.000 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 94/134 2023-07-14T11:49:43.108 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-07-14T11:49:43.272 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 95/134 2023-07-14T11:49:43.346 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 95/134 2023-07-14T11:49:43.390 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-07-14T11:49:43.507 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 96/134 2023-07-14T11:49:43.683 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-07-14T11:49:43.898 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-14T11:49:44.132 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-14T11:49:44.424 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-14T11:49:44.663 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-14T11:49:44.886 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-14T11:49:45.045 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-14T11:49:45.334 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-14T11:49:45.550 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-14T11:49:45.782 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-14T11:49:45.855 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 96/134 2023-07-14T11:49:45.855 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /sys 2023-07-14T11:49:45.855 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /proc 2023-07-14T11:49:45.855 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /dev 2023-07-14T11:49:45.855 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /run 2023-07-14T11:49:45.856 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /mnt 2023-07-14T11:49:45.856 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /var/tmp 2023-07-14T11:49:45.856 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /home 2023-07-14T11:49:45.856 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /tmp 2023-07-14T11:49:45.856 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /dev 2023-07-14T11:49:45.856 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-14T11:49:45.984 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-14T11:49:46.196 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-14T11:49:46.217 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 97/134 2023-07-14T11:49:46.461 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-14T11:49:46.694 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-14T11:49:46.904 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-14T11:49:47.197 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-14T11:49:47.464 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-14T11:49:47.660 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-14T11:49:47.779 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 97/134 2023-07-14T11:49:47.946 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-14T11:49:47.995 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 98/134 2023-07-14T11:49:48.213 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 98/134 2023-07-14T11:49:48.418 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 99/134 2023-07-14T11:49:48.870 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 99/134 2023-07-14T11:49:49.025 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 100/134 2023-07-14T11:49:49.042 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 100/134 2023-07-14T11:49:49.435 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-14T11:49:49.735 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-14T11:49:50.037 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-14T11:49:50.579 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-14T11:49:50.715 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 101/134 2023-07-14T11:49:50.926 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-14T11:49:50.955 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 101/134 2023-07-14T11:49:51.097 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4757.g3fc0089c.e 102/134 2023-07-14T11:49:51.179 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-14T11:49:51.314 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4757.g3fc0089c. 103/134 2023-07-14T11:49:51.541 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noa 104/134 2023-07-14T11:49:51.755 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noa 104/134 2023-07-14T11:49:51.952 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.n 105/134 2023-07-14T11:49:52.084 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-14T11:49:52.270 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 92/134 2023-07-14T11:49:52.361 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 92/134 2023-07-14T11:49:52.612 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-rgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 93/134 2023-07-14T11:49:52.910 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.n 105/134 2023-07-14T11:49:53.091 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc 106/134 2023-07-14T11:49:53.182 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc 106/134 2023-07-14T11:49:53.531 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4757.g3fc0089c.el 107/134 2023-07-14T11:49:53.790 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 108/134 2023-07-14T11:49:53.826 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 108/134 2023-07-14T11:49:53.990 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 109/134 2023-07-14T11:49:54.002 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 109/134 2023-07-14T11:49:55.300 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-2:18.0.0-4757.g3fc0089c.el8.x86_64 110/134 2023-07-14T11:49:55.490 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 111/134 2023-07-14T11:49:55.523 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 111/134 2023-07-14T11:49:55.707 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4757.g3fc00 112/134 2023-07-14T11:49:55.749 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4757.g3fc00 112/134 2023-07-14T11:49:55.940 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 113/134 2023-07-14T11:49:56.285 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 113/134 2023-07-14T11:49:56.524 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 114/134 2023-07-14T11:49:57.744 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 94/134 2023-07-14T11:49:57.961 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 94/134 2023-07-14T11:49:58.710 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 94/134 2023-07-14T11:49:59.033 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 95/134 2023-07-14T11:49:59.169 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 95/134 2023-07-14T11:49:59.310 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 96/134 2023-07-14T11:50:01.079 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 114/134 2023-07-14T11:50:01.388 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-test-2:18.0.0-4757.g3fc0089c.el8.x86_64 115/134 2023-07-14T11:50:01.539 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-07-14T11:50:01.731 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-07-14T11:50:02.135 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 96/134 2023-07-14T11:50:02.135 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /sys 2023-07-14T11:50:02.135 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /proc 2023-07-14T11:50:02.135 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /dev 2023-07-14T11:50:02.135 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /run 2023-07-14T11:50:02.136 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /mnt 2023-07-14T11:50:02.136 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /var/tmp 2023-07-14T11:50:02.136 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /home 2023-07-14T11:50:02.136 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /tmp 2023-07-14T11:50:02.137 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /dev 2023-07-14T11:50:02.137 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-14T11:50:02.471 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 97/134 2023-07-14T11:50:02.609 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-07-14T11:50:02.946 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-07-14T11:50:03.134 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 119/134 2023-07-14T11:50:03.319 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-14T11:50:03.535 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephfs-devel-2:18.0.0-4757.g3fc0089c.el8.x86_ 121/134 2023-07-14T11:50:03.699 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-14T11:50:04.032 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 97/134 2023-07-14T11:50:04.177 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 98/134 2023-07-14T11:50:04.394 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 98/134 2023-07-14T11:50:04.587 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 99/134 2023-07-14T11:50:05.039 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 99/134 2023-07-14T11:50:05.188 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 100/134 2023-07-14T11:50:05.204 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 100/134 2023-07-14T11:50:05.888 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-14T11:50:06.156 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-14T11:50:06.175 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-14T11:50:06.398 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-top-2:18.0.0-4757.g3fc0089c.el8.noarch 124/134 2023-07-14T11:50:06.647 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 125/134 2023-07-14T11:50:06.817 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 101/134 2023-07-14T11:50:06.952 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-nbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 126/134 2023-07-14T11:50:07.021 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 101/134 2023-07-14T11:50:07.135 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4757.g3fc0089c.e 102/134 2023-07-14T11:50:07.307 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-14T11:50:07.320 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4757.g3fc0089c. 103/134 2023-07-14T11:50:07.456 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-14T11:50:07.464 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-14T11:50:07.512 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noa 104/134 2023-07-14T11:50:07.665 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-07-14T11:50:07.723 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noa 104/134 2023-07-14T11:50:07.872 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-07-14T11:50:07.955 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-07-14T11:50:07.985 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.n 105/134 2023-07-14T11:50:08.155 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-07-14T11:50:08.168 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-07-14T11:50:08.168 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-14T11:50:08.425 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-14T11:50:08.425 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-14T11:50:08.652 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-14T11:50:08.879 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-14T11:50:08.940 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.n 105/134 2023-07-14T11:50:09.103 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc 106/134 2023-07-14T11:50:09.194 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc 106/134 2023-07-14T11:50:09.396 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-14T11:50:09.397 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-2:18.0.0-4757.g3fc0089c.el8.x86_64 1/134 2023-07-14T11:50:09.397 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 2/134 2023-07-14T11:50:09.397 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 3/134 2023-07-14T11:50:09.397 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 4/134 2023-07-14T11:50:09.397 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4757.g3fc00 5/134 2023-07-14T11:50:09.397 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 6/134 2023-07-14T11:50:09.397 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 7/134 2023-07-14T11:50:09.398 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 8/134 2023-07-14T11:50:09.398 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 9/134 2023-07-14T11:50:09.398 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 10/134 2023-07-14T11:50:09.398 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 11/134 2023-07-14T11:50:09.398 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-test-2:18.0.0-4757.g3fc0089c.el8.x86_64 12/134 2023-07-14T11:50:09.398 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 13/134 2023-07-14T11:50:09.398 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephfs-devel-2:18.0.0-4757.g3fc0089c.el8.x86_ 14/134 2023-07-14T11:50:09.399 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 15/134 2023-07-14T11:50:09.399 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 16/134 2023-07-14T11:50:09.399 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados-devel-2:18.0.0-4757.g3fc0089c.el8.x86_6 17/134 2023-07-14T11:50:09.399 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86 18/134 2023-07-14T11:50:09.399 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 19/134 2023-07-14T11:50:09.399 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4757.g3fc0089c.el 20/134 2023-07-14T11:50:09.399 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ceph-common-2:18.0.0-4757.g3fc0089c.el8. 21/134 2023-07-14T11:50:09.399 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cephfs-2:18.0.0-4757.g3fc0089c.el8.x86_6 22/134 2023-07-14T11:50:09.400 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rados-2:18.0.0-4757.g3fc0089c.el8.x86_64 23/134 2023-07-14T11:50:09.400 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 24/134 2023-07-14T11:50:09.400 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 25/134 2023-07-14T11:50:09.400 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 26/134 2023-07-14T11:50:09.400 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 27/134 2023-07-14T11:50:09.400 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-nbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 28/134 2023-07-14T11:50:09.400 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4757.g3fc0089c. 29/134 2023-07-14T11:50:09.400 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noa 30/134 2023-07-14T11:50:09.401 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.n 31/134 2023-07-14T11:50:09.408 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc 32/134 2023-07-14T11:50:09.409 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4757.g3fc0089c.el 33/134 2023-07-14T11:50:09.409 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 34/134 2023-07-14T11:50:09.409 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4757.g3fc0089c.e 35/134 2023-07-14T11:50:09.409 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 36/134 2023-07-14T11:50:09.409 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 37/134 2023-07-14T11:50:09.409 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-top-2:18.0.0-4757.g3fc0089c.el8.noarch 38/134 2023-07-14T11:50:09.409 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-14T11:50:09.410 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-14T11:50:09.410 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-14T11:50:09.410 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-14T11:50:09.410 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-14T11:50:09.410 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-14T11:50:09.410 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-14T11:50:09.410 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-14T11:50:09.410 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-14T11:50:09.411 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-14T11:50:09.411 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-14T11:50:09.411 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-14T11:50:09.411 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-14T11:50:09.411 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-14T11:50:09.411 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-14T11:50:09.411 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-14T11:50:09.412 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-14T11:50:09.412 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-14T11:50:09.412 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-14T11:50:09.412 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-14T11:50:09.412 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-14T11:50:09.412 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-14T11:50:09.412 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-14T11:50:09.412 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-14T11:50:09.413 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-14T11:50:09.413 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-14T11:50:09.413 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-14T11:50:09.413 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-14T11:50:09.413 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-14T11:50:09.413 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-14T11:50:09.413 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-14T11:50:09.414 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-14T11:50:09.414 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-14T11:50:09.414 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-07-14T11:50:09.414 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-07-14T11:50:09.414 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-07-14T11:50:09.414 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-07-14T11:50:09.414 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-07-14T11:50:09.414 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-07-14T11:50:09.415 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-07-14T11:50:09.415 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-07-14T11:50:09.415 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-07-14T11:50:09.415 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-07-14T11:50:09.415 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-07-14T11:50:09.415 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-07-14T11:50:09.415 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-07-14T11:50:09.415 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-07-14T11:50:09.416 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-07-14T11:50:09.416 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-07-14T11:50:09.416 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-07-14T11:50:09.416 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-07-14T11:50:09.416 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-07-14T11:50:09.416 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-07-14T11:50:09.416 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-07-14T11:50:09.417 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-07-14T11:50:09.417 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-07-14T11:50:09.417 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-07-14T11:50:09.417 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-07-14T11:50:09.417 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-07-14T11:50:09.417 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-07-14T11:50:09.417 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-07-14T11:50:09.417 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-07-14T11:50:09.418 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-07-14T11:50:09.418 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-07-14T11:50:09.418 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-07-14T11:50:09.418 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-07-14T11:50:09.418 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-07-14T11:50:09.418 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-07-14T11:50:09.418 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-07-14T11:50:09.418 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-07-14T11:50:09.419 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-07-14T11:50:09.419 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-07-14T11:50:09.419 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-07-14T11:50:09.419 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-07-14T11:50:09.419 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-07-14T11:50:09.419 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-07-14T11:50:09.419 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-07-14T11:50:09.419 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-07-14T11:50:09.420 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-07-14T11:50:09.420 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-07-14T11:50:09.420 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-07-14T11:50:09.420 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-14T11:50:09.420 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-07-14T11:50:09.420 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-14T11:50:09.420 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-07-14T11:50:09.421 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-07-14T11:50:09.421 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-07-14T11:50:09.421 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-07-14T11:50:09.421 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-07-14T11:50:09.421 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-14T11:50:09.421 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 129/134 2023-07-14T11:50:09.421 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-14T11:50:09.421 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 131/134 2023-07-14T11:50:09.422 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-14T11:50:09.422 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-14T11:50:09.587 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4757.g3fc0089c.el 107/134 2023-07-14T11:50:09.776 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 108/134 2023-07-14T11:50:09.811 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 108/134 2023-07-14T11:50:09.970 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 109/134 2023-07-14T11:50:09.981 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 109/134 2023-07-14T11:50:11.258 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-2:18.0.0-4757.g3fc0089c.el8.x86_64 110/134 2023-07-14T11:50:11.456 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 111/134 2023-07-14T11:50:11.489 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 111/134 2023-07-14T11:50:11.638 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4757.g3fc00 112/134 2023-07-14T11:50:11.677 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4757.g3fc00 112/134 2023-07-14T11:50:11.828 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 113/134 2023-07-14T11:50:11.867 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-14T11:50:11.867 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:51 ago on Fri 14 Jul 2023 11:48:20 AM UTC. 2023-07-14T11:50:12.171 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 113/134 2023-07-14T11:50:12.320 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 114/134 2023-07-14T11:50:14.405 INFO:teuthology.orchestra.run.smithi097.stdout:Installed products updated. 2023-07-14T11:50:14.617 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-14T11:50:14.617 INFO:teuthology.orchestra.run.smithi097.stdout:Upgraded: 2023-07-14T11:50:14.618 INFO:teuthology.orchestra.run.smithi097.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-14T11:50:14.618 INFO:teuthology.orchestra.run.smithi097.stdout: librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:14.618 INFO:teuthology.orchestra.run.smithi097.stdout: librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:14.618 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2023-07-14T11:50:14.618 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-14T11:50:14.618 INFO:teuthology.orchestra.run.smithi097.stdout: automake-1.16.1-7.el8.noarch 2023-07-14T11:50:14.618 INFO:teuthology.orchestra.run.smithi097.stdout: bison-3.0.4-10.el8.x86_64 2023-07-14T11:50:14.619 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:14.619 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:14.619 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:14.619 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:14.619 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-grafana-dashboards-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:50:14.619 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-immutable-object-cache-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:14.619 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:14.620 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:14.620 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:50:14.620 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:50:14.620 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:50:14.620 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:50:14.620 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:50:14.620 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:14.620 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:14.621 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-prometheus-alerts-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:50:14.621 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:14.621 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:14.621 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-test-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:14.621 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:50:14.621 INFO:teuthology.orchestra.run.smithi097.stdout: cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:50:14.621 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:14.622 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-top-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:50:14.622 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-14T11:50:14.622 INFO:teuthology.orchestra.run.smithi097.stdout: flex-2.6.1-9.el8.x86_64 2023-07-14T11:50:14.622 INFO:teuthology.orchestra.run.smithi097.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-14T11:50:14.622 INFO:teuthology.orchestra.run.smithi097.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-14T11:50:14.622 INFO:teuthology.orchestra.run.smithi097.stdout: jq-1.6-3.el8.x86_64 2023-07-14T11:50:14.622 INFO:teuthology.orchestra.run.smithi097.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-14T11:50:14.622 INFO:teuthology.orchestra.run.smithi097.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-14T11:50:14.623 INFO:teuthology.orchestra.run.smithi097.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-14T11:50:14.623 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs-devel-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:14.623 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:14.623 INFO:teuthology.orchestra.run.smithi097.stdout: libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:14.623 INFO:teuthology.orchestra.run.smithi097.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-14T11:50:14.623 INFO:teuthology.orchestra.run.smithi097.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-14T11:50:14.623 INFO:teuthology.orchestra.run.smithi097.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-14T11:50:14.623 INFO:teuthology.orchestra.run.smithi097.stdout: libpq-13.5-1.el8.x86_64 2023-07-14T11:50:14.624 INFO:teuthology.orchestra.run.smithi097.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-14T11:50:14.624 INFO:teuthology.orchestra.run.smithi097.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-14T11:50:14.624 INFO:teuthology.orchestra.run.smithi097.stdout: librados-devel-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:14.624 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:14.624 INFO:teuthology.orchestra.run.smithi097.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-14T11:50:14.624 INFO:teuthology.orchestra.run.smithi097.stdout: librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:14.624 INFO:teuthology.orchestra.run.smithi097.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-14T11:50:14.625 INFO:teuthology.orchestra.run.smithi097.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-14T11:50:14.625 INFO:teuthology.orchestra.run.smithi097.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-14T11:50:14.625 INFO:teuthology.orchestra.run.smithi097.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-14T11:50:14.625 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-14T11:50:14.625 INFO:teuthology.orchestra.run.smithi097.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-14T11:50:14.625 INFO:teuthology.orchestra.run.smithi097.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-14T11:50:14.625 INFO:teuthology.orchestra.run.smithi097.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-14T11:50:14.625 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-14T11:50:14.626 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-14T11:50:14.626 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-14T11:50:14.626 INFO:teuthology.orchestra.run.smithi097.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-14T11:50:14.626 INFO:teuthology.orchestra.run.smithi097.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-14T11:50:14.626 INFO:teuthology.orchestra.run.smithi097.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-14T11:50:14.626 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-14T11:50:14.626 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-14T11:50:14.626 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-14T11:50:14.627 INFO:teuthology.orchestra.run.smithi097.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-14T11:50:14.627 INFO:teuthology.orchestra.run.smithi097.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-14T11:50:14.627 INFO:teuthology.orchestra.run.smithi097.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-14T11:50:14.627 INFO:teuthology.orchestra.run.smithi097.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-14T11:50:14.627 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-14T11:50:14.627 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-argparse-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:14.628 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:14.628 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cephfs-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:14.628 INFO:teuthology.orchestra.run.smithi097.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-14T11:50:14.628 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-14T11:50:14.628 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-14T11:50:14.628 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-14T11:50:14.628 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-14T11:50:14.629 INFO:teuthology.orchestra.run.smithi097.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-14T11:50:14.629 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-14T11:50:14.629 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-14T11:50:14.629 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-14T11:50:14.629 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-14T11:50:14.629 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-14T11:50:14.629 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-14T11:50:14.630 INFO:teuthology.orchestra.run.smithi097.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-14T11:50:14.630 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-14T11:50:14.630 INFO:teuthology.orchestra.run.smithi097.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-14T11:50:14.630 INFO:teuthology.orchestra.run.smithi097.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-14T11:50:14.630 INFO:teuthology.orchestra.run.smithi097.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-14T11:50:14.630 INFO:teuthology.orchestra.run.smithi097.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-14T11:50:14.630 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-14T11:50:14.630 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-14T11:50:14.631 INFO:teuthology.orchestra.run.smithi097.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-14T11:50:14.631 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-14T11:50:14.631 INFO:teuthology.orchestra.run.smithi097.stdout: python3-portend-2.6-1.el8.noarch 2023-07-14T11:50:14.631 INFO:teuthology.orchestra.run.smithi097.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-14T11:50:14.631 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-14T11:50:14.631 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-14T11:50:14.631 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-14T11:50:14.631 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-14T11:50:14.632 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-14T11:50:14.632 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rados-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:14.632 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:14.632 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-14T11:50:14.632 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-14T11:50:14.632 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:14.632 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-14T11:50:14.633 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-14T11:50:14.633 INFO:teuthology.orchestra.run.smithi097.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-14T11:50:14.633 INFO:teuthology.orchestra.run.smithi097.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-14T11:50:14.633 INFO:teuthology.orchestra.run.smithi097.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-14T11:50:14.633 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-14T11:50:14.633 INFO:teuthology.orchestra.run.smithi097.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-14T11:50:14.633 INFO:teuthology.orchestra.run.smithi097.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-14T11:50:14.633 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-14T11:50:14.634 INFO:teuthology.orchestra.run.smithi097.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-14T11:50:14.634 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-14T11:50:14.634 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-14T11:50:14.634 INFO:teuthology.orchestra.run.smithi097.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-14T11:50:14.634 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:14.634 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:14.634 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-nbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:14.634 INFO:teuthology.orchestra.run.smithi097.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-14T11:50:14.635 INFO:teuthology.orchestra.run.smithi097.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-14T11:50:14.635 INFO:teuthology.orchestra.run.smithi097.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-14T11:50:14.635 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-14T11:50:14.635 INFO:teuthology.orchestra.run.smithi097.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-14T11:50:14.635 INFO:teuthology.orchestra.run.smithi097.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-14T11:50:14.636 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-14T11:50:14.636 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2023-07-14T11:50:14.834 DEBUG:teuthology.parallel:result is None 2023-07-14T11:50:16.878 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 114/134 2023-07-14T11:50:17.784 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-test-2:18.0.0-4757.g3fc0089c.el8.x86_64 115/134 2023-07-14T11:50:18.083 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-07-14T11:50:18.246 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 117/134 2023-07-14T11:50:18.496 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-07-14T11:50:18.677 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-07-14T11:50:18.865 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-07-14T11:50:19.106 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-07-14T11:50:19.256 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libcephfs-devel-2:18.0.0-4757.g3fc0089c.el8.x86_ 121/134 2023-07-14T11:50:19.445 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-14T11:50:20.678 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 96/134 2023-07-14T11:50:20.678 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /sys 2023-07-14T11:50:20.678 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /proc 2023-07-14T11:50:20.678 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /dev 2023-07-14T11:50:20.678 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /run 2023-07-14T11:50:20.679 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /mnt 2023-07-14T11:50:20.679 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /var/tmp 2023-07-14T11:50:20.679 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /home 2023-07-14T11:50:20.679 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /tmp 2023-07-14T11:50:20.679 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /dev 2023-07-14T11:50:20.679 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:50:21.133 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 97/134 2023-07-14T11:50:21.680 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-14T11:50:21.952 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-14T11:50:21.969 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-14T11:50:22.155 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : cephfs-top-2:18.0.0-4757.g3fc0089c.el8.noarch 124/134 2023-07-14T11:50:22.367 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : rbd-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 125/134 2023-07-14T11:50:22.672 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : rbd-nbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 126/134 2023-07-14T11:50:22.972 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 97/134 2023-07-14T11:50:23.049 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-14T11:50:23.187 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 98/134 2023-07-14T11:50:23.201 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-07-14T11:50:23.285 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-07-14T11:50:23.472 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 98/134 2023-07-14T11:50:23.501 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-07-14T11:50:23.513 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-07-14T11:50:23.699 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 99/134 2023-07-14T11:50:23.701 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-14T11:50:23.708 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-14T11:50:23.845 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-07-14T11:50:23.845 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-14T11:50:24.013 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-14T11:50:24.013 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-14T11:50:24.222 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-14T11:50:24.260 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 99/134 2023-07-14T11:50:24.391 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-14T11:50:24.482 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 100/134 2023-07-14T11:50:24.526 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 100/134 2023-07-14T11:50:24.885 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-14T11:50:24.886 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-2:18.0.0-4757.g3fc0089c.el8.x86_64 1/134 2023-07-14T11:50:24.886 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 2/134 2023-07-14T11:50:24.886 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 3/134 2023-07-14T11:50:24.886 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 4/134 2023-07-14T11:50:24.886 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4757.g3fc00 5/134 2023-07-14T11:50:24.886 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 6/134 2023-07-14T11:50:24.887 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 7/134 2023-07-14T11:50:24.887 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 8/134 2023-07-14T11:50:24.887 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 9/134 2023-07-14T11:50:24.887 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 10/134 2023-07-14T11:50:24.887 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 11/134 2023-07-14T11:50:24.887 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-test-2:18.0.0-4757.g3fc0089c.el8.x86_64 12/134 2023-07-14T11:50:24.888 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 13/134 2023-07-14T11:50:24.888 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libcephfs-devel-2:18.0.0-4757.g3fc0089c.el8.x86_ 14/134 2023-07-14T11:50:24.888 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 15/134 2023-07-14T11:50:24.888 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 16/134 2023-07-14T11:50:24.888 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librados-devel-2:18.0.0-4757.g3fc0089c.el8.x86_6 17/134 2023-07-14T11:50:24.888 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86 18/134 2023-07-14T11:50:24.889 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 19/134 2023-07-14T11:50:24.889 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4757.g3fc0089c.el 20/134 2023-07-14T11:50:24.889 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-ceph-common-2:18.0.0-4757.g3fc0089c.el8. 21/134 2023-07-14T11:50:24.889 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-cephfs-2:18.0.0-4757.g3fc0089c.el8.x86_6 22/134 2023-07-14T11:50:24.889 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-rados-2:18.0.0-4757.g3fc0089c.el8.x86_64 23/134 2023-07-14T11:50:24.889 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-rbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 24/134 2023-07-14T11:50:24.889 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-rgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 25/134 2023-07-14T11:50:24.890 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : rbd-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 26/134 2023-07-14T11:50:24.890 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 27/134 2023-07-14T11:50:24.890 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : rbd-nbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 28/134 2023-07-14T11:50:24.890 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4757.g3fc0089c. 29/134 2023-07-14T11:50:24.898 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noa 30/134 2023-07-14T11:50:24.898 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.n 31/134 2023-07-14T11:50:24.898 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc 32/134 2023-07-14T11:50:24.898 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4757.g3fc0089c.el 33/134 2023-07-14T11:50:24.898 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 34/134 2023-07-14T11:50:24.899 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4757.g3fc0089c.e 35/134 2023-07-14T11:50:24.899 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 36/134 2023-07-14T11:50:24.899 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 37/134 2023-07-14T11:50:24.899 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : cephfs-top-2:18.0.0-4757.g3fc0089c.el8.noarch 38/134 2023-07-14T11:50:24.899 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-14T11:50:24.899 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-14T11:50:24.899 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-14T11:50:24.900 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-14T11:50:24.900 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-14T11:50:24.900 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-14T11:50:24.900 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-14T11:50:24.900 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-14T11:50:24.900 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-14T11:50:24.901 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-14T11:50:24.901 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-14T11:50:24.901 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-14T11:50:24.901 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-14T11:50:24.901 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-14T11:50:24.901 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-14T11:50:24.901 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-14T11:50:24.902 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-14T11:50:24.902 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-14T11:50:24.902 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-14T11:50:24.902 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-14T11:50:24.902 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-14T11:50:24.902 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-14T11:50:24.902 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-14T11:50:24.902 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-14T11:50:24.903 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-14T11:50:24.903 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-14T11:50:24.903 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-14T11:50:24.903 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-14T11:50:24.903 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-14T11:50:24.903 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-14T11:50:24.903 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-14T11:50:24.904 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-14T11:50:24.904 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-14T11:50:24.904 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-07-14T11:50:24.904 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-07-14T11:50:24.904 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-07-14T11:50:24.904 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-07-14T11:50:24.905 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-07-14T11:50:24.905 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-07-14T11:50:24.905 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-07-14T11:50:24.905 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-07-14T11:50:24.905 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-07-14T11:50:24.905 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-07-14T11:50:24.905 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-07-14T11:50:24.906 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-07-14T11:50:24.906 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-07-14T11:50:24.906 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-07-14T11:50:24.906 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-07-14T11:50:24.906 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-07-14T11:50:24.906 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-07-14T11:50:24.906 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-07-14T11:50:24.907 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-07-14T11:50:24.907 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-07-14T11:50:24.907 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-07-14T11:50:24.907 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-07-14T11:50:24.907 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-07-14T11:50:24.907 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-07-14T11:50:24.907 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-07-14T11:50:24.908 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-07-14T11:50:24.908 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-07-14T11:50:24.908 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-07-14T11:50:24.908 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-07-14T11:50:24.908 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-07-14T11:50:24.909 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-07-14T11:50:24.909 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-07-14T11:50:24.909 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-07-14T11:50:24.910 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-07-14T11:50:24.910 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-07-14T11:50:24.910 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-07-14T11:50:24.911 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-07-14T11:50:24.911 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-07-14T11:50:24.911 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-07-14T11:50:24.911 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-07-14T11:50:24.912 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-07-14T11:50:24.912 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-07-14T11:50:24.912 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-07-14T11:50:24.913 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-07-14T11:50:24.913 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-07-14T11:50:24.913 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-07-14T11:50:24.914 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-07-14T11:50:24.914 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-07-14T11:50:24.914 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-07-14T11:50:24.915 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-07-14T11:50:24.915 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-07-14T11:50:24.915 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-07-14T11:50:24.916 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-07-14T11:50:24.916 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-07-14T11:50:24.916 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-07-14T11:50:24.916 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-07-14T11:50:24.917 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-07-14T11:50:24.917 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 129/134 2023-07-14T11:50:24.917 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-14T11:50:24.917 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 131/134 2023-07-14T11:50:24.918 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-14T11:50:24.918 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-14T11:50:27.291 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 101/134 2023-07-14T11:50:27.324 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-14T11:50:27.324 INFO:teuthology.orchestra.run.smithi093.stdout:Last metadata expiration check: 0:01:54 ago on Fri 14 Jul 2023 11:48:33 AM UTC. 2023-07-14T11:50:27.490 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 101/134 2023-07-14T11:50:27.623 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4757.g3fc0089c.e 102/134 2023-07-14T11:50:27.880 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4757.g3fc0089c. 103/134 2023-07-14T11:50:28.105 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noa 104/134 2023-07-14T11:50:28.381 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noa 104/134 2023-07-14T11:50:28.595 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.n 105/134 2023-07-14T11:50:29.784 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.n 105/134 2023-07-14T11:50:29.891 INFO:teuthology.orchestra.run.smithi093.stdout:Installed products updated. 2023-07-14T11:50:29.956 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc 106/134 2023-07-14T11:50:30.092 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc 106/134 2023-07-14T11:50:30.102 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-14T11:50:30.102 INFO:teuthology.orchestra.run.smithi093.stdout:Upgraded: 2023-07-14T11:50:30.103 INFO:teuthology.orchestra.run.smithi093.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-14T11:50:30.103 INFO:teuthology.orchestra.run.smithi093.stdout: librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:30.103 INFO:teuthology.orchestra.run.smithi093.stdout: librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:30.103 INFO:teuthology.orchestra.run.smithi093.stdout:Installed: 2023-07-14T11:50:30.103 INFO:teuthology.orchestra.run.smithi093.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-14T11:50:30.103 INFO:teuthology.orchestra.run.smithi093.stdout: automake-1.16.1-7.el8.noarch 2023-07-14T11:50:30.103 INFO:teuthology.orchestra.run.smithi093.stdout: bison-3.0.4-10.el8.x86_64 2023-07-14T11:50:30.104 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:30.104 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:30.104 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:30.104 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:30.104 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-grafana-dashboards-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:50:30.104 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-immutable-object-cache-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:30.104 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:30.105 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:30.105 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:50:30.105 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:50:30.105 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:50:30.105 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-modules-core-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:50:30.105 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:50:30.105 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:30.105 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:30.106 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-prometheus-alerts-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:50:30.106 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:30.106 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:30.106 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-test-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:30.106 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:50:30.107 INFO:teuthology.orchestra.run.smithi093.stdout: cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:50:30.107 INFO:teuthology.orchestra.run.smithi093.stdout: cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:30.107 INFO:teuthology.orchestra.run.smithi093.stdout: cephfs-top-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:50:30.107 INFO:teuthology.orchestra.run.smithi093.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-14T11:50:30.107 INFO:teuthology.orchestra.run.smithi093.stdout: flex-2.6.1-9.el8.x86_64 2023-07-14T11:50:30.107 INFO:teuthology.orchestra.run.smithi093.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-14T11:50:30.107 INFO:teuthology.orchestra.run.smithi093.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-14T11:50:30.108 INFO:teuthology.orchestra.run.smithi093.stdout: jq-1.6-3.el8.x86_64 2023-07-14T11:50:30.108 INFO:teuthology.orchestra.run.smithi093.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-14T11:50:30.108 INFO:teuthology.orchestra.run.smithi093.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-14T11:50:30.108 INFO:teuthology.orchestra.run.smithi093.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-14T11:50:30.108 INFO:teuthology.orchestra.run.smithi093.stdout: libcephfs-devel-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:30.108 INFO:teuthology.orchestra.run.smithi093.stdout: libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:30.109 INFO:teuthology.orchestra.run.smithi093.stdout: libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:30.109 INFO:teuthology.orchestra.run.smithi093.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-14T11:50:30.109 INFO:teuthology.orchestra.run.smithi093.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-14T11:50:30.109 INFO:teuthology.orchestra.run.smithi093.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-14T11:50:30.110 INFO:teuthology.orchestra.run.smithi093.stdout: libpq-13.5-1.el8.x86_64 2023-07-14T11:50:30.110 INFO:teuthology.orchestra.run.smithi093.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-14T11:50:30.110 INFO:teuthology.orchestra.run.smithi093.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-14T11:50:30.110 INFO:teuthology.orchestra.run.smithi093.stdout: librados-devel-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:30.112 INFO:teuthology.orchestra.run.smithi093.stdout: libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:30.112 INFO:teuthology.orchestra.run.smithi093.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-14T11:50:30.112 INFO:teuthology.orchestra.run.smithi093.stdout: librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:30.112 INFO:teuthology.orchestra.run.smithi093.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-14T11:50:30.112 INFO:teuthology.orchestra.run.smithi093.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-14T11:50:30.113 INFO:teuthology.orchestra.run.smithi093.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-14T11:50:30.113 INFO:teuthology.orchestra.run.smithi093.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-14T11:50:30.113 INFO:teuthology.orchestra.run.smithi093.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-14T11:50:30.113 INFO:teuthology.orchestra.run.smithi093.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-14T11:50:30.113 INFO:teuthology.orchestra.run.smithi093.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-14T11:50:30.113 INFO:teuthology.orchestra.run.smithi093.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-14T11:50:30.113 INFO:teuthology.orchestra.run.smithi093.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-14T11:50:30.114 INFO:teuthology.orchestra.run.smithi093.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-14T11:50:30.114 INFO:teuthology.orchestra.run.smithi093.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-14T11:50:30.114 INFO:teuthology.orchestra.run.smithi093.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-14T11:50:30.114 INFO:teuthology.orchestra.run.smithi093.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-14T11:50:30.114 INFO:teuthology.orchestra.run.smithi093.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-14T11:50:30.114 INFO:teuthology.orchestra.run.smithi093.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-14T11:50:30.114 INFO:teuthology.orchestra.run.smithi093.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-14T11:50:30.114 INFO:teuthology.orchestra.run.smithi093.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-14T11:50:30.115 INFO:teuthology.orchestra.run.smithi093.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-14T11:50:30.115 INFO:teuthology.orchestra.run.smithi093.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-14T11:50:30.115 INFO:teuthology.orchestra.run.smithi093.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-14T11:50:30.115 INFO:teuthology.orchestra.run.smithi093.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-14T11:50:30.115 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-14T11:50:30.115 INFO:teuthology.orchestra.run.smithi093.stdout: python3-ceph-argparse-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:30.116 INFO:teuthology.orchestra.run.smithi093.stdout: python3-ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:30.116 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cephfs-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:30.116 INFO:teuthology.orchestra.run.smithi093.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-14T11:50:30.116 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-14T11:50:30.116 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-14T11:50:30.116 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-14T11:50:30.116 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-14T11:50:30.117 INFO:teuthology.orchestra.run.smithi093.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-14T11:50:30.117 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-14T11:50:30.117 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-14T11:50:30.117 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-14T11:50:30.117 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-14T11:50:30.117 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-14T11:50:30.117 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-14T11:50:30.118 INFO:teuthology.orchestra.run.smithi093.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-14T11:50:30.118 INFO:teuthology.orchestra.run.smithi093.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-14T11:50:30.118 INFO:teuthology.orchestra.run.smithi093.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-14T11:50:30.118 INFO:teuthology.orchestra.run.smithi093.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-14T11:50:30.118 INFO:teuthology.orchestra.run.smithi093.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-14T11:50:30.118 INFO:teuthology.orchestra.run.smithi093.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-14T11:50:30.118 INFO:teuthology.orchestra.run.smithi093.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-14T11:50:30.118 INFO:teuthology.orchestra.run.smithi093.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-14T11:50:30.119 INFO:teuthology.orchestra.run.smithi093.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-14T11:50:30.119 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-14T11:50:30.119 INFO:teuthology.orchestra.run.smithi093.stdout: python3-portend-2.6-1.el8.noarch 2023-07-14T11:50:30.119 INFO:teuthology.orchestra.run.smithi093.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-14T11:50:30.119 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-14T11:50:30.119 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-14T11:50:30.120 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-14T11:50:30.120 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-14T11:50:30.120 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-14T11:50:30.120 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rados-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:30.120 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:30.120 INFO:teuthology.orchestra.run.smithi093.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-14T11:50:30.121 INFO:teuthology.orchestra.run.smithi093.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-14T11:50:30.121 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:30.121 INFO:teuthology.orchestra.run.smithi093.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-14T11:50:30.121 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-14T11:50:30.121 INFO:teuthology.orchestra.run.smithi093.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-14T11:50:30.121 INFO:teuthology.orchestra.run.smithi093.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-14T11:50:30.121 INFO:teuthology.orchestra.run.smithi093.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-14T11:50:30.122 INFO:teuthology.orchestra.run.smithi093.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-14T11:50:30.122 INFO:teuthology.orchestra.run.smithi093.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-14T11:50:30.122 INFO:teuthology.orchestra.run.smithi093.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-14T11:50:30.122 INFO:teuthology.orchestra.run.smithi093.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-14T11:50:30.122 INFO:teuthology.orchestra.run.smithi093.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-14T11:50:30.122 INFO:teuthology.orchestra.run.smithi093.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-14T11:50:30.122 INFO:teuthology.orchestra.run.smithi093.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-14T11:50:30.123 INFO:teuthology.orchestra.run.smithi093.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-14T11:50:30.123 INFO:teuthology.orchestra.run.smithi093.stdout: rbd-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:30.123 INFO:teuthology.orchestra.run.smithi093.stdout: rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:30.123 INFO:teuthology.orchestra.run.smithi093.stdout: rbd-nbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:30.123 INFO:teuthology.orchestra.run.smithi093.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-14T11:50:30.123 INFO:teuthology.orchestra.run.smithi093.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-14T11:50:30.123 INFO:teuthology.orchestra.run.smithi093.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-14T11:50:30.124 INFO:teuthology.orchestra.run.smithi093.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-14T11:50:30.124 INFO:teuthology.orchestra.run.smithi093.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-14T11:50:30.125 INFO:teuthology.orchestra.run.smithi093.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-14T11:50:30.125 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-14T11:50:30.125 INFO:teuthology.orchestra.run.smithi093.stdout:Complete! 2023-07-14T11:50:30.307 DEBUG:teuthology.parallel:result is None 2023-07-14T11:50:30.521 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4757.g3fc0089c.el 107/134 2023-07-14T11:50:30.777 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 108/134 2023-07-14T11:50:30.845 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 108/134 2023-07-14T11:50:31.081 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 109/134 2023-07-14T11:50:31.111 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 109/134 2023-07-14T11:50:32.599 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-2:18.0.0-4757.g3fc0089c.el8.x86_64 110/134 2023-07-14T11:50:32.846 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 111/134 2023-07-14T11:50:32.921 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 111/134 2023-07-14T11:50:33.139 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4757.g3fc00 112/134 2023-07-14T11:50:33.221 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4757.g3fc00 112/134 2023-07-14T11:50:33.398 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 113/134 2023-07-14T11:50:33.830 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 113/134 2023-07-14T11:50:34.040 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 114/134 2023-07-14T11:50:39.411 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 114/134 2023-07-14T11:50:40.749 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-test-2:18.0.0-4757.g3fc0089c.el8.x86_64 115/134 2023-07-14T11:50:41.164 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-07-14T11:50:41.512 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 117/134 2023-07-14T11:50:41.803 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-07-14T11:50:41.979 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-07-14T11:50:42.248 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-07-14T11:50:42.493 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-07-14T11:50:42.743 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libcephfs-devel-2:18.0.0-4757.g3fc0089c.el8.x86_ 121/134 2023-07-14T11:50:42.914 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-14T11:50:46.447 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-14T11:50:46.758 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-14T11:50:46.796 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-14T11:50:46.997 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : cephfs-top-2:18.0.0-4757.g3fc0089c.el8.noarch 124/134 2023-07-14T11:50:47.273 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : rbd-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 125/134 2023-07-14T11:50:47.622 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : rbd-nbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 126/134 2023-07-14T11:50:47.996 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-14T11:50:48.186 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-07-14T11:50:48.299 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-07-14T11:50:48.529 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-07-14T11:50:48.558 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-07-14T11:50:48.795 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-14T11:50:48.818 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-14T11:50:48.965 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-07-14T11:50:48.966 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-14T11:50:49.227 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-14T11:50:49.228 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-14T11:50:49.494 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-14T11:50:49.737 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-14T11:50:50.001 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-14T11:50:50.001 INFO:teuthology.orchestra.run.smithi017.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-07-14T11:50:50.001 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:50:50.657 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-2:18.0.0-4757.g3fc0089c.el8.x86_64 1/134 2023-07-14T11:50:50.657 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 2/134 2023-07-14T11:50:50.657 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 3/134 2023-07-14T11:50:50.658 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 4/134 2023-07-14T11:50:50.658 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4757.g3fc00 5/134 2023-07-14T11:50:50.658 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 6/134 2023-07-14T11:50:50.658 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 7/134 2023-07-14T11:50:50.658 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 8/134 2023-07-14T11:50:50.658 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 9/134 2023-07-14T11:50:50.658 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 10/134 2023-07-14T11:50:50.659 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 11/134 2023-07-14T11:50:50.659 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-test-2:18.0.0-4757.g3fc0089c.el8.x86_64 12/134 2023-07-14T11:50:50.659 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 13/134 2023-07-14T11:50:50.659 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libcephfs-devel-2:18.0.0-4757.g3fc0089c.el8.x86_ 14/134 2023-07-14T11:50:50.659 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 15/134 2023-07-14T11:50:50.659 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 16/134 2023-07-14T11:50:50.659 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librados-devel-2:18.0.0-4757.g3fc0089c.el8.x86_6 17/134 2023-07-14T11:50:50.659 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86 18/134 2023-07-14T11:50:50.660 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 19/134 2023-07-14T11:50:50.660 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4757.g3fc0089c.el 20/134 2023-07-14T11:50:50.660 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-ceph-common-2:18.0.0-4757.g3fc0089c.el8. 21/134 2023-07-14T11:50:50.660 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-cephfs-2:18.0.0-4757.g3fc0089c.el8.x86_6 22/134 2023-07-14T11:50:50.660 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-rados-2:18.0.0-4757.g3fc0089c.el8.x86_64 23/134 2023-07-14T11:50:50.660 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-rbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 24/134 2023-07-14T11:50:50.661 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-rgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 25/134 2023-07-14T11:50:50.661 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : rbd-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 26/134 2023-07-14T11:50:50.661 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 27/134 2023-07-14T11:50:50.661 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : rbd-nbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 28/134 2023-07-14T11:50:50.661 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4757.g3fc0089c. 29/134 2023-07-14T11:50:50.668 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noa 30/134 2023-07-14T11:50:50.668 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.n 31/134 2023-07-14T11:50:50.668 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc 32/134 2023-07-14T11:50:50.669 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4757.g3fc0089c.el 33/134 2023-07-14T11:50:50.669 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 34/134 2023-07-14T11:50:50.669 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4757.g3fc0089c.e 35/134 2023-07-14T11:50:50.669 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 36/134 2023-07-14T11:50:50.669 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 37/134 2023-07-14T11:50:50.669 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : cephfs-top-2:18.0.0-4757.g3fc0089c.el8.noarch 38/134 2023-07-14T11:50:50.669 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-14T11:50:50.670 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-14T11:50:50.670 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-14T11:50:50.670 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-14T11:50:50.670 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-14T11:50:50.670 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-14T11:50:50.670 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-14T11:50:50.670 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-14T11:50:50.670 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-14T11:50:50.671 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-14T11:50:50.671 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-14T11:50:50.671 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-14T11:50:50.671 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-14T11:50:50.671 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-14T11:50:50.671 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-14T11:50:50.671 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-14T11:50:50.671 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-14T11:50:50.672 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-14T11:50:50.672 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-14T11:50:50.672 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-14T11:50:50.672 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-14T11:50:50.672 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-14T11:50:50.672 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-14T11:50:50.672 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-14T11:50:50.673 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-14T11:50:50.673 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-14T11:50:50.673 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-14T11:50:50.673 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-14T11:50:50.673 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-14T11:50:50.673 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-14T11:50:50.673 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-14T11:50:50.674 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-14T11:50:50.674 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-14T11:50:50.674 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-07-14T11:50:50.674 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-07-14T11:50:50.674 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-07-14T11:50:50.674 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-07-14T11:50:50.674 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-07-14T11:50:50.674 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-07-14T11:50:50.675 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-07-14T11:50:50.675 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-07-14T11:50:50.675 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-07-14T11:50:50.675 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-07-14T11:50:50.675 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-07-14T11:50:50.675 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-07-14T11:50:50.675 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-07-14T11:50:50.675 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-07-14T11:50:50.676 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-07-14T11:50:50.676 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-07-14T11:50:50.676 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-07-14T11:50:50.676 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-07-14T11:50:50.676 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-07-14T11:50:50.677 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-07-14T11:50:50.677 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-07-14T11:50:50.677 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-07-14T11:50:50.677 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-07-14T11:50:50.677 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-07-14T11:50:50.677 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-07-14T11:50:50.677 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-07-14T11:50:50.677 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-07-14T11:50:50.678 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-07-14T11:50:50.678 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-07-14T11:50:50.678 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-07-14T11:50:50.678 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-07-14T11:50:50.678 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-07-14T11:50:50.678 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-07-14T11:50:50.679 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-07-14T11:50:50.679 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-07-14T11:50:50.679 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-07-14T11:50:50.679 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-07-14T11:50:50.679 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-07-14T11:50:50.679 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-07-14T11:50:50.679 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-07-14T11:50:50.680 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-07-14T11:50:50.680 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-07-14T11:50:50.680 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-07-14T11:50:50.680 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-07-14T11:50:50.680 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-07-14T11:50:50.680 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-07-14T11:50:50.680 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-07-14T11:50:50.680 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-07-14T11:50:50.681 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-07-14T11:50:50.681 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-07-14T11:50:50.681 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-07-14T11:50:50.681 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-07-14T11:50:50.681 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-07-14T11:50:50.681 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-07-14T11:50:50.681 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-07-14T11:50:50.681 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-07-14T11:50:50.682 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-07-14T11:50:50.682 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 129/134 2023-07-14T11:50:50.682 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-14T11:50:50.682 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 131/134 2023-07-14T11:50:50.682 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-14T11:50:50.682 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-14T11:50:53.505 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-14T11:50:53.505 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:02:17 ago on Fri 14 Jul 2023 11:48:36 AM UTC. 2023-07-14T11:50:56.511 INFO:teuthology.orchestra.run.smithi017.stdout:Installed products updated. 2023-07-14T11:50:56.755 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:50:56.756 INFO:teuthology.orchestra.run.smithi017.stdout:Upgraded: 2023-07-14T11:50:56.756 INFO:teuthology.orchestra.run.smithi017.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-14T11:50:56.756 INFO:teuthology.orchestra.run.smithi017.stdout: librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:56.756 INFO:teuthology.orchestra.run.smithi017.stdout: librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:56.756 INFO:teuthology.orchestra.run.smithi017.stdout:Installed: 2023-07-14T11:50:56.756 INFO:teuthology.orchestra.run.smithi017.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-14T11:50:56.757 INFO:teuthology.orchestra.run.smithi017.stdout: automake-1.16.1-7.el8.noarch 2023-07-14T11:50:56.757 INFO:teuthology.orchestra.run.smithi017.stdout: bison-3.0.4-10.el8.x86_64 2023-07-14T11:50:56.757 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:56.757 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:56.758 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:56.758 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:56.758 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-grafana-dashboards-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:50:56.758 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-immutable-object-cache-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:56.758 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:56.758 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:56.759 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:50:56.759 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:50:56.759 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:50:56.759 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-modules-core-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:50:56.759 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:50:56.759 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:56.759 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:56.759 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-prometheus-alerts-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:50:56.760 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:56.760 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:56.760 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-test-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:56.760 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:50:56.760 INFO:teuthology.orchestra.run.smithi017.stdout: cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:50:56.760 INFO:teuthology.orchestra.run.smithi017.stdout: cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:56.760 INFO:teuthology.orchestra.run.smithi017.stdout: cephfs-top-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T11:50:56.761 INFO:teuthology.orchestra.run.smithi017.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-14T11:50:56.761 INFO:teuthology.orchestra.run.smithi017.stdout: flex-2.6.1-9.el8.x86_64 2023-07-14T11:50:56.761 INFO:teuthology.orchestra.run.smithi017.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-14T11:50:56.761 INFO:teuthology.orchestra.run.smithi017.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-14T11:50:56.761 INFO:teuthology.orchestra.run.smithi017.stdout: jq-1.6-3.el8.x86_64 2023-07-14T11:50:56.761 INFO:teuthology.orchestra.run.smithi017.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-14T11:50:56.761 INFO:teuthology.orchestra.run.smithi017.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-14T11:50:56.762 INFO:teuthology.orchestra.run.smithi017.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-14T11:50:56.762 INFO:teuthology.orchestra.run.smithi017.stdout: libcephfs-devel-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:56.762 INFO:teuthology.orchestra.run.smithi017.stdout: libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:56.762 INFO:teuthology.orchestra.run.smithi017.stdout: libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:56.762 INFO:teuthology.orchestra.run.smithi017.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-14T11:50:56.762 INFO:teuthology.orchestra.run.smithi017.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-14T11:50:56.762 INFO:teuthology.orchestra.run.smithi017.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-14T11:50:56.763 INFO:teuthology.orchestra.run.smithi017.stdout: libpq-13.5-1.el8.x86_64 2023-07-14T11:50:56.763 INFO:teuthology.orchestra.run.smithi017.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-14T11:50:56.763 INFO:teuthology.orchestra.run.smithi017.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-14T11:50:56.763 INFO:teuthology.orchestra.run.smithi017.stdout: librados-devel-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:56.763 INFO:teuthology.orchestra.run.smithi017.stdout: libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:56.763 INFO:teuthology.orchestra.run.smithi017.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-14T11:50:56.763 INFO:teuthology.orchestra.run.smithi017.stdout: librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:56.764 INFO:teuthology.orchestra.run.smithi017.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-14T11:50:56.764 INFO:teuthology.orchestra.run.smithi017.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-14T11:50:56.764 INFO:teuthology.orchestra.run.smithi017.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-14T11:50:56.764 INFO:teuthology.orchestra.run.smithi017.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-14T11:50:56.764 INFO:teuthology.orchestra.run.smithi017.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-14T11:50:56.764 INFO:teuthology.orchestra.run.smithi017.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-14T11:50:56.764 INFO:teuthology.orchestra.run.smithi017.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-14T11:50:56.765 INFO:teuthology.orchestra.run.smithi017.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-14T11:50:56.765 INFO:teuthology.orchestra.run.smithi017.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-14T11:50:56.765 INFO:teuthology.orchestra.run.smithi017.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-14T11:50:56.765 INFO:teuthology.orchestra.run.smithi017.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-14T11:50:56.765 INFO:teuthology.orchestra.run.smithi017.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-14T11:50:56.765 INFO:teuthology.orchestra.run.smithi017.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-14T11:50:56.766 INFO:teuthology.orchestra.run.smithi017.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-14T11:50:56.766 INFO:teuthology.orchestra.run.smithi017.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-14T11:50:56.766 INFO:teuthology.orchestra.run.smithi017.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-14T11:50:56.766 INFO:teuthology.orchestra.run.smithi017.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-14T11:50:56.766 INFO:teuthology.orchestra.run.smithi017.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-14T11:50:56.766 INFO:teuthology.orchestra.run.smithi017.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-14T11:50:56.766 INFO:teuthology.orchestra.run.smithi017.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-14T11:50:56.766 INFO:teuthology.orchestra.run.smithi017.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-14T11:50:56.767 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-14T11:50:56.767 INFO:teuthology.orchestra.run.smithi017.stdout: python3-ceph-argparse-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:56.767 INFO:teuthology.orchestra.run.smithi017.stdout: python3-ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:56.767 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cephfs-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:56.767 INFO:teuthology.orchestra.run.smithi017.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-14T11:50:56.767 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-14T11:50:56.767 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-14T11:50:56.768 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-14T11:50:56.768 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-14T11:50:56.768 INFO:teuthology.orchestra.run.smithi017.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-14T11:50:56.768 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-14T11:50:56.768 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-14T11:50:56.768 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-14T11:50:56.768 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-14T11:50:56.769 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-14T11:50:56.769 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-14T11:50:56.769 INFO:teuthology.orchestra.run.smithi017.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-14T11:50:56.769 INFO:teuthology.orchestra.run.smithi017.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-14T11:50:56.769 INFO:teuthology.orchestra.run.smithi017.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-14T11:50:56.769 INFO:teuthology.orchestra.run.smithi017.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-14T11:50:56.769 INFO:teuthology.orchestra.run.smithi017.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-14T11:50:56.769 INFO:teuthology.orchestra.run.smithi017.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-14T11:50:56.770 INFO:teuthology.orchestra.run.smithi017.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-14T11:50:56.770 INFO:teuthology.orchestra.run.smithi017.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-14T11:50:56.770 INFO:teuthology.orchestra.run.smithi017.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-14T11:50:56.770 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-14T11:50:56.770 INFO:teuthology.orchestra.run.smithi017.stdout: python3-portend-2.6-1.el8.noarch 2023-07-14T11:50:56.770 INFO:teuthology.orchestra.run.smithi017.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-14T11:50:56.770 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-14T11:50:56.770 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-14T11:50:56.771 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-14T11:50:56.771 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-14T11:50:56.771 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-14T11:50:56.771 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rados-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:56.771 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:56.771 INFO:teuthology.orchestra.run.smithi017.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-14T11:50:56.771 INFO:teuthology.orchestra.run.smithi017.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-14T11:50:56.771 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:56.772 INFO:teuthology.orchestra.run.smithi017.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-14T11:50:56.772 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-14T11:50:56.772 INFO:teuthology.orchestra.run.smithi017.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-14T11:50:56.772 INFO:teuthology.orchestra.run.smithi017.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-14T11:50:56.772 INFO:teuthology.orchestra.run.smithi017.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-14T11:50:56.772 INFO:teuthology.orchestra.run.smithi017.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-14T11:50:56.773 INFO:teuthology.orchestra.run.smithi017.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-14T11:50:56.773 INFO:teuthology.orchestra.run.smithi017.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-14T11:50:56.773 INFO:teuthology.orchestra.run.smithi017.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-14T11:50:56.773 INFO:teuthology.orchestra.run.smithi017.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-14T11:50:56.773 INFO:teuthology.orchestra.run.smithi017.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-14T11:50:56.773 INFO:teuthology.orchestra.run.smithi017.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-14T11:50:56.773 INFO:teuthology.orchestra.run.smithi017.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-14T11:50:56.773 INFO:teuthology.orchestra.run.smithi017.stdout: rbd-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:56.774 INFO:teuthology.orchestra.run.smithi017.stdout: rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:56.774 INFO:teuthology.orchestra.run.smithi017.stdout: rbd-nbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T11:50:56.774 INFO:teuthology.orchestra.run.smithi017.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-14T11:50:56.774 INFO:teuthology.orchestra.run.smithi017.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-14T11:50:56.774 INFO:teuthology.orchestra.run.smithi017.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-14T11:50:56.774 INFO:teuthology.orchestra.run.smithi017.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-14T11:50:56.774 INFO:teuthology.orchestra.run.smithi017.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-14T11:50:56.775 INFO:teuthology.orchestra.run.smithi017.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-14T11:50:56.775 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:50:56.775 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2023-07-14T11:50:57.106 DEBUG:teuthology.parallel:result is None 2023-07-14T11:50:57.108 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-14T11:50:57.692 DEBUG:teuthology.orchestra.run.smithi017:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-14T11:50:57.749 INFO:teuthology.orchestra.run.smithi017.stdout:18.0.0-4757.g3fc0089c.el8 2023-07-14T11:50:57.750 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4757.g3fc0089c.el8 2023-07-14T11:50:57.750 INFO:teuthology.task.install:The correct ceph version 18.0.0-4757.g3fc0089c is installed. 2023-07-14T11:50:57.751 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-14T11:50:58.325 DEBUG:teuthology.orchestra.run.smithi093:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-14T11:50:58.354 INFO:teuthology.orchestra.run.smithi093.stdout:18.0.0-4757.g3fc0089c.el8 2023-07-14T11:50:58.355 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4757.g3fc0089c.el8 2023-07-14T11:50:58.355 INFO:teuthology.task.install:The correct ceph version 18.0.0-4757.g3fc0089c is installed. 2023-07-14T11:50:58.356 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-14T11:50:58.944 DEBUG:teuthology.orchestra.run.smithi097:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-14T11:50:58.974 INFO:teuthology.orchestra.run.smithi097.stdout:18.0.0-4757.g3fc0089c.el8 2023-07-14T11:50:58.974 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4757.g3fc0089c.el8 2023-07-14T11:50:58.974 INFO:teuthology.task.install:The correct ceph version 18.0.0-4757.g3fc0089c is installed. 2023-07-14T11:50:58.975 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-07-14T11:50:58.975 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2023-07-14T11:50:58.976 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-14T11:50:59.074 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-07-14T11:50:59.074 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-14T11:50:59.122 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-14T11:50:59.122 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-14T11:50:59.170 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-07-14T11:50:59.170 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2023-07-14T11:50:59.170 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/usr/bin/daemon-helper 2023-07-14T11:50:59.241 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-14T11:50:59.349 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-07-14T11:50:59.350 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/usr/bin/daemon-helper 2023-07-14T11:50:59.383 DEBUG:teuthology.orchestra.run.smithi093:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-14T11:50:59.455 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-14T11:50:59.456 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/daemon-helper 2023-07-14T11:50:59.488 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-14T11:50:59.557 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-07-14T11:50:59.558 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2023-07-14T11:50:59.558 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-14T11:50:59.629 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-14T11:50:59.737 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-07-14T11:50:59.738 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-14T11:50:59.771 DEBUG:teuthology.orchestra.run.smithi093:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-14T11:50:59.842 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-14T11:50:59.843 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-14T11:50:59.874 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-14T11:50:59.944 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-07-14T11:50:59.945 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2023-07-14T11:50:59.946 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/usr/bin/stdin-killer 2023-07-14T11:51:00.016 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-14T11:51:00.124 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-07-14T11:51:00.124 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/usr/bin/stdin-killer 2023-07-14T11:51:00.158 DEBUG:teuthology.orchestra.run.smithi093:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-14T11:51:00.230 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-14T11:51:00.230 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/stdin-killer 2023-07-14T11:51:00.260 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-14T11:51:00.329 INFO:teuthology.run_tasks:Running task cephadm... 2023-07-14T11:51:00.427 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '3fc0089cc1082313a98853d48b892acdc1385b8b'} 2023-07-14T11:51:00.428 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-14T11:51:00.438 INFO:tasks.cephadm:Cluster fsid is b3c386cc-223c-11ee-9b33-001a4aab830c 2023-07-14T11:51:00.439 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-07-14T11:51:00.439 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.17', 'mon.b': '172.21.15.93', 'mon.c': '172.21.15.97'} 2023-07-14T11:51:00.440 INFO:tasks.cephadm:First mon is mon.a on smithi017 2023-07-14T11:51:00.440 INFO:tasks.cephadm:First mgr is x 2023-07-14T11:51:00.440 INFO:tasks.cephadm:Normalizing hostnames... 2023-07-14T11:51:00.441 DEBUG:teuthology.orchestra.run.smithi017:> sudo hostname $(hostname -s) 2023-07-14T11:51:00.514 DEBUG:teuthology.orchestra.run.smithi093:> sudo hostname $(hostname -s) 2023-07-14T11:51:00.547 DEBUG:teuthology.orchestra.run.smithi097:> sudo hostname $(hostname -s) 2023-07-14T11:51:00.578 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-07-14T11:51:00.579 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-14T11:51:01.191 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '3fc0089cc1082313a98853d48b892acdc1385b8b', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/71399/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-4757-g3fc0089c', 'node_name': '172.21.2.8+braggi08', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-4757.g3fc0089c'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-07-13 19:47:46.444052', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-2023Jul13', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2023-07-14T11:51:01.838 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-rishabh-2023Jul13, sha1 3fc0089cc1082313a98853d48b892acdc1385b8b from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-14T11:51:01.838 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/x86_64/flavors/default/cephadm 2023-07-14T11:51:01.839 DEBUG:teuthology.orchestra.run.smithi017:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-07-14T11:51:02.734 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-rw-r--. 1 ubuntu ubuntu 663791 Jul 14 11:51 /home/ubuntu/cephtest/cephadm 2023-07-14T11:51:02.735 DEBUG:teuthology.orchestra.run.smithi093:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-07-14T11:51:03.899 INFO:teuthology.orchestra.run.smithi093.stdout:-rw-rw-r--. 1 ubuntu ubuntu 663791 Jul 14 11:51 /home/ubuntu/cephtest/cephadm 2023-07-14T11:51:03.900 DEBUG:teuthology.orchestra.run.smithi097:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-07-14T11:51:04.841 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-rw-r--. 1 ubuntu ubuntu 663791 Jul 14 11:51 /home/ubuntu/cephtest/cephadm 2023-07-14T11:51:04.842 DEBUG:teuthology.orchestra.run.smithi017:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-07-14T11:51:04.893 DEBUG:teuthology.orchestra.run.smithi093:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-07-14T11:51:04.916 DEBUG:teuthology.orchestra.run.smithi097:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-07-14T11:51:04.951 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b on all hosts... 2023-07-14T11:51:04.952 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b pull 2023-07-14T11:51:04.958 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b pull 2023-07-14T11:51:04.960 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b pull 2023-07-14T11:51:06.251 INFO:teuthology.orchestra.run.smithi093.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b... 2023-07-14T11:51:06.255 INFO:teuthology.orchestra.run.smithi097.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b... 2023-07-14T11:51:09.308 INFO:teuthology.orchestra.run.smithi017.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b... 2023-07-14T11:51:36.781 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2023-07-14T11:51:36.782 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-14T11:51:36.782 INFO:teuthology.orchestra.run.smithi017.stdout: "image_id": "afd12e232ec04c7e77932076425916985d97c3486288fff2ba931630e84a9295", 2023-07-14T11:51:36.782 INFO:teuthology.orchestra.run.smithi017.stdout: "repo_digests": [ 2023-07-14T11:51:36.782 INFO:teuthology.orchestra.run.smithi017.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c4bf3497f7cb9cae2d0db734c6fa7ef173cd8dc715907bde16e4a68082d82e1d" 2023-07-14T11:51:36.782 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2023-07-14T11:51:36.782 INFO:teuthology.orchestra.run.smithi017.stdout:} 2023-07-14T11:51:48.532 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-07-14T11:51:48.533 INFO:teuthology.orchestra.run.smithi093.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-14T11:51:48.533 INFO:teuthology.orchestra.run.smithi093.stdout: "image_id": "afd12e232ec04c7e77932076425916985d97c3486288fff2ba931630e84a9295", 2023-07-14T11:51:48.533 INFO:teuthology.orchestra.run.smithi093.stdout: "repo_digests": [ 2023-07-14T11:51:48.533 INFO:teuthology.orchestra.run.smithi093.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c4bf3497f7cb9cae2d0db734c6fa7ef173cd8dc715907bde16e4a68082d82e1d" 2023-07-14T11:51:48.533 INFO:teuthology.orchestra.run.smithi093.stdout: ] 2023-07-14T11:51:48.533 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-07-14T11:51:49.131 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-07-14T11:51:49.132 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-14T11:51:49.132 INFO:teuthology.orchestra.run.smithi097.stdout: "image_id": "afd12e232ec04c7e77932076425916985d97c3486288fff2ba931630e84a9295", 2023-07-14T11:51:49.132 INFO:teuthology.orchestra.run.smithi097.stdout: "repo_digests": [ 2023-07-14T11:51:49.132 INFO:teuthology.orchestra.run.smithi097.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c4bf3497f7cb9cae2d0db734c6fa7ef173cd8dc715907bde16e4a68082d82e1d" 2023-07-14T11:51:49.132 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2023-07-14T11:51:49.132 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-07-14T11:51:49.153 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /etc/ceph 2023-07-14T11:51:49.236 DEBUG:teuthology.orchestra.run.smithi093:> sudo mkdir -p /etc/ceph 2023-07-14T11:51:49.274 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /etc/ceph 2023-07-14T11:51:49.305 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 777 /etc/ceph 2023-07-14T11:51:49.376 DEBUG:teuthology.orchestra.run.smithi093:> sudo chmod 777 /etc/ceph 2023-07-14T11:51:49.408 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 777 /etc/ceph 2023-07-14T11:51:49.435 INFO:tasks.cephadm:Writing seed config... 2023-07-14T11:51:49.436 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-07-14T11:51:49.436 INFO:tasks.cephadm: override: [client] debug client = 20 2023-07-14T11:51:49.437 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-07-14T11:51:49.437 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-07-14T11:51:49.437 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-07-14T11:51:49.437 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-07-14T11:51:49.437 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-07-14T11:51:49.438 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2023-07-14T11:51:49.438 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-07-14T11:51:49.439 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2023-07-14T11:51:49.439 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2023-07-14T11:51:49.440 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2023-07-14T11:51:49.440 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2023-07-14T11:51:49.440 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-07-14T11:51:49.441 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-07-14T11:51:49.441 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-07-14T11:51:49.442 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-07-14T11:51:49.442 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-07-14T11:51:49.443 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-07-14T11:51:49.443 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-07-14T11:51:49.444 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-07-14T11:51:49.444 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2023-07-14T11:51:49.445 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-07-14T11:51:49.445 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-07-14T11:51:49.446 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-07-14T11:51:49.446 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-07-14T11:51:49.447 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-07-14T11:51:49.447 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-07-14T11:51:49.447 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-07-14T11:51:49.447 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2023-07-14T11:51:49.448 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-07-14T11:51:49.448 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2023-07-14T11:51:49.449 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2023-07-14T11:51:49.449 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-07-14T11:51:49.450 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-07-14T11:51:49.450 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2023-07-14T11:51:49.450 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-07-14T11:51:49.450 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-07-14T11:51:49.451 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-07-14T11:51:49.451 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-07-14T11:51:49.451 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-07-14T11:51:49.451 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-07-14T11:51:49.451 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-07-14T11:51:49.452 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2023-07-14T11:51:49.452 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2023-07-14T11:51:49.453 DEBUG:teuthology.orchestra.run.smithi017:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-07-14T11:51:49.496 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = b3c386cc-223c-11ee-9b33-001a4aab830c ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-07-14T11:51:49.497 DEBUG:teuthology.orchestra.run.smithi017:mon.a> sudo journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mon.a.service 2023-07-14T11:51:49.541 DEBUG:teuthology.orchestra.run.smithi017:mgr.x> sudo journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mgr.x.service 2023-07-14T11:51:49.586 INFO:tasks.cephadm:Bootstrapping... 2023-07-14T11:51:49.586 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b -v bootstrap --fsid b3c386cc-223c-11ee-9b33-001a4aab830c --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.17 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-07-14T11:51:49.609 INFO:journalctl@ceph.mon.a.smithi017.stdout:-- Logs begin at Fri 2023-07-14 11:32:03 UTC. -- 2023-07-14T11:51:49.655 INFO:journalctl@ceph.mgr.x.smithi017.stdout:-- Logs begin at Fri 2023-07-14 11:32:03 UTC. -- 2023-07-14T11:51:49.816 INFO:teuthology.orchestra.run.smithi017.stdout:-------------------------------------------------------------------------------- 2023-07-14T11:51:49.816 INFO:teuthology.orchestra.run.smithi017.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b', '-v', 'bootstrap', '--fsid', 'b3c386cc-223c-11ee-9b33-001a4aab830c', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.17', '--skip-admin-label'] 2023-07-14T11:51:49.991 INFO:teuthology.orchestra.run.smithi017.stdout:/bin/podman: stdout 4.1.1 2023-07-14T11:51:49.991 INFO:teuthology.orchestra.run.smithi017.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-07-14T11:51:49.992 INFO:teuthology.orchestra.run.smithi017.stdout:Verifying podman|docker is present... 2023-07-14T11:51:50.164 INFO:teuthology.orchestra.run.smithi017.stdout:/bin/podman: stdout 4.1.1 2023-07-14T11:51:50.164 INFO:teuthology.orchestra.run.smithi017.stdout:Verifying lvm2 is present... 2023-07-14T11:51:50.165 INFO:teuthology.orchestra.run.smithi017.stdout:Verifying time synchronization is in place... 2023-07-14T11:51:50.179 INFO:teuthology.orchestra.run.smithi017.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-07-14T11:51:50.180 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-07-14T11:51:50.193 INFO:teuthology.orchestra.run.smithi017.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-07-14T11:51:50.193 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stdout inactive 2023-07-14T11:51:50.210 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stdout enabled 2023-07-14T11:51:50.225 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stdout active 2023-07-14T11:51:50.225 INFO:teuthology.orchestra.run.smithi017.stdout:Unit chronyd.service is enabled and running 2023-07-14T11:51:50.226 INFO:teuthology.orchestra.run.smithi017.stdout:Repeating the final host check... 2023-07-14T11:51:50.398 INFO:teuthology.orchestra.run.smithi017.stdout:/bin/podman: stdout 4.1.1 2023-07-14T11:51:50.399 INFO:teuthology.orchestra.run.smithi017.stdout:podman (/bin/podman) version 4.1.1 is present 2023-07-14T11:51:50.399 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl is present 2023-07-14T11:51:50.399 INFO:teuthology.orchestra.run.smithi017.stdout:lvcreate is present 2023-07-14T11:51:50.412 INFO:teuthology.orchestra.run.smithi017.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-07-14T11:51:50.412 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-07-14T11:51:50.425 INFO:teuthology.orchestra.run.smithi017.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-07-14T11:51:50.425 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stdout inactive 2023-07-14T11:51:50.440 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stdout enabled 2023-07-14T11:51:50.453 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stdout active 2023-07-14T11:51:50.454 INFO:teuthology.orchestra.run.smithi017.stdout:Unit chronyd.service is enabled and running 2023-07-14T11:51:50.454 INFO:teuthology.orchestra.run.smithi017.stdout:Host looks OK 2023-07-14T11:51:50.454 INFO:teuthology.orchestra.run.smithi017.stdout:Cluster fsid: b3c386cc-223c-11ee-9b33-001a4aab830c 2023-07-14T11:51:50.454 INFO:teuthology.orchestra.run.smithi017.stdout:Acquiring lock 140049103370224 on /run/cephadm/b3c386cc-223c-11ee-9b33-001a4aab830c.lock 2023-07-14T11:51:50.455 INFO:teuthology.orchestra.run.smithi017.stdout:Lock 140049103370224 acquired on /run/cephadm/b3c386cc-223c-11ee-9b33-001a4aab830c.lock 2023-07-14T11:51:50.455 INFO:teuthology.orchestra.run.smithi017.stdout:Verifying IP 172.21.15.17 port 3300 ... 2023-07-14T11:51:50.456 INFO:teuthology.orchestra.run.smithi017.stdout:Verifying IP 172.21.15.17 port 6789 ... 2023-07-14T11:51:50.456 INFO:teuthology.orchestra.run.smithi017.stdout:Base mon IP(s) is [172.21.15.17:3300, 172.21.15.17:6789], mon addrv is [v2:172.21.15.17:3300,v1:172.21.15.17:6789] 2023-07-14T11:51:50.461 INFO:teuthology.orchestra.run.smithi017.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp metric 100 2023-07-14T11:51:50.462 INFO:teuthology.orchestra.run.smithi017.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.17 metric 100 2023-07-14T11:51:50.468 INFO:teuthology.orchestra.run.smithi017.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-07-14T11:51:50.468 INFO:teuthology.orchestra.run.smithi017.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2023-07-14T11:51:50.469 INFO:teuthology.orchestra.run.smithi017.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1795sec hoplimit 64 pref medium 2023-07-14T11:51:50.475 INFO:teuthology.orchestra.run.smithi017.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-07-14T11:51:50.476 INFO:teuthology.orchestra.run.smithi017.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-07-14T11:51:50.476 INFO:teuthology.orchestra.run.smithi017.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-07-14T11:51:50.476 INFO:teuthology.orchestra.run.smithi017.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2023-07-14T11:51:50.476 INFO:teuthology.orchestra.run.smithi017.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe59:d992/64 scope link 2023-07-14T11:51:50.476 INFO:teuthology.orchestra.run.smithi017.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-07-14T11:51:50.477 INFO:teuthology.orchestra.run.smithi017.stdout:Mon IP `172.21.15.17` is in CIDR network `172.21.0.0/20` 2023-07-14T11:51:50.477 INFO:teuthology.orchestra.run.smithi017.stdout:Mon IP `172.21.15.17` is in CIDR network `172.21.0.0/20` 2023-07-14T11:51:50.477 INFO:teuthology.orchestra.run.smithi017.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-07-14T11:51:50.478 INFO:teuthology.orchestra.run.smithi017.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-07-14T11:51:50.478 INFO:teuthology.orchestra.run.smithi017.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b... 2023-07-14T11:51:51.455 INFO:teuthology.orchestra.run.smithi017.stdout:/bin/podman: stdout afd12e232ec04c7e77932076425916985d97c3486288fff2ba931630e84a9295 2023-07-14T11:51:51.456 INFO:teuthology.orchestra.run.smithi017.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b... 2023-07-14T11:51:51.456 INFO:teuthology.orchestra.run.smithi017.stdout:/bin/podman: stderr Getting image source signatures 2023-07-14T11:51:51.456 INFO:teuthology.orchestra.run.smithi017.stdout:/bin/podman: stderr Copying blob sha256:d8eac6e0ff3c58bd5607bffacb1c2214d20cdd3ff8cfaf23cfeb0cc60b2d0d6b 2023-07-14T11:51:51.456 INFO:teuthology.orchestra.run.smithi017.stdout:/bin/podman: stderr Copying blob sha256:6022c611d474f2b306ddd1edfc8360123ece5266119096f25fde843d7967e636 2023-07-14T11:51:51.456 INFO:teuthology.orchestra.run.smithi017.stdout:/bin/podman: stderr Copying config sha256:afd12e232ec04c7e77932076425916985d97c3486288fff2ba931630e84a9295 2023-07-14T11:51:51.457 INFO:teuthology.orchestra.run.smithi017.stdout:/bin/podman: stderr Writing manifest to image destination 2023-07-14T11:51:51.457 INFO:teuthology.orchestra.run.smithi017.stdout:/bin/podman: stderr Storing signatures 2023-07-14T11:51:52.438 INFO:teuthology.orchestra.run.smithi017.stdout:ceph: stdout ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev) 2023-07-14T11:51:52.438 INFO:teuthology.orchestra.run.smithi017.stdout:Ceph version: ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev) 2023-07-14T11:51:52.438 INFO:teuthology.orchestra.run.smithi017.stdout:Extracting ceph user uid/gid from container image... 2023-07-14T11:51:53.212 INFO:teuthology.orchestra.run.smithi017.stdout:stat: stdout 167 167 2023-07-14T11:51:53.213 INFO:teuthology.orchestra.run.smithi017.stdout:Creating initial keys... 2023-07-14T11:51:54.287 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph-authtool: stdout AQDZNrFkiEp9MBAAitmpDTB/cWa4qfZpSwKLHA== 2023-07-14T11:51:55.163 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph-authtool: stdout AQDaNrFkMZJDKhAAD6LcCDcp5Y+lInd8AG5jNA== 2023-07-14T11:51:56.140 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph-authtool: stdout AQDbNrFk5edEJhAAlFjcZhTBpAP6dOpo7haCsg== 2023-07-14T11:51:56.141 INFO:teuthology.orchestra.run.smithi017.stdout:Creating initial monmap... 2023-07-14T11:51:56.918 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-07-14T11:51:56.919 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-07-14T11:51:56.919 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to b3c386cc-223c-11ee-9b33-001a4aab830c 2023-07-14T11:51:56.919 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-07-14T11:51:56.919 INFO:teuthology.orchestra.run.smithi017.stdout:monmaptool for a [v2:172.21.15.17:3300,v1:172.21.15.17:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-07-14T11:51:56.919 INFO:teuthology.orchestra.run.smithi017.stdout:setting min_mon_release = pacific 2023-07-14T11:51:56.919 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/monmaptool: set fsid to b3c386cc-223c-11ee-9b33-001a4aab830c 2023-07-14T11:51:56.919 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-07-14T11:51:56.920 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:51:56.920 INFO:teuthology.orchestra.run.smithi017.stdout:Creating mon... 2023-07-14T11:51:57.897 INFO:teuthology.orchestra.run.smithi017.stdout:create mon.a on 2023-07-14T11:51:58.172 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-07-14T11:51:58.383 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-07-14T11:51:58.642 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-b3c386cc-223c-11ee-9b33-001a4aab830c.target → /etc/systemd/system/ceph-b3c386cc-223c-11ee-9b33-001a4aab830c.target. 2023-07-14T11:51:58.643 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-b3c386cc-223c-11ee-9b33-001a4aab830c.target → /etc/systemd/system/ceph-b3c386cc-223c-11ee-9b33-001a4aab830c.target. 2023-07-14T11:51:58.985 INFO:teuthology.orchestra.run.smithi017.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mon.a 2023-07-14T11:51:58.985 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mon.a.service: Unit ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mon.a.service not loaded. 2023-07-14T11:51:59.190 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b3c386cc-223c-11ee-9b33-001a4aab830c.target.wants/ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mon.a.service → /etc/systemd/system/ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@.service. 2023-07-14T11:51:59.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:51:59 smithi017 systemd[1]: Starting Ceph mon.a for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T11:51:59.979 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:51:59 smithi017 podman[125788]: 2023-07-14T11:51:59.980 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:51:59 smithi017 bash[125788]: 392aa2925303642217eb7584173d38fb3f8cb9868def65267c0186b35f28f88b 2023-07-14T11:52:00.010 INFO:teuthology.orchestra.run.smithi017.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-14T11:52:00.010 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stdout disabled 2023-07-14T11:52:00.026 INFO:teuthology.orchestra.run.smithi017.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-14T11:52:00.026 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stdout inactive 2023-07-14T11:52:00.026 INFO:teuthology.orchestra.run.smithi017.stdout:firewalld.service is not enabled 2023-07-14T11:52:00.026 INFO:teuthology.orchestra.run.smithi017.stdout:Not possible to enable service . firewalld.service is not available 2023-07-14T11:52:00.027 INFO:teuthology.orchestra.run.smithi017.stdout:Waiting for mon to start... 2023-07-14T11:52:00.027 INFO:teuthology.orchestra.run.smithi017.stdout:Waiting for mon... 2023-07-14T11:52:00.308 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:51:59 smithi017 ceph-mon[125828]: mkfs b3c386cc-223c-11ee-9b33-001a4aab830c 2023-07-14T11:52:00.308 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:00 smithi017 ceph-mon[125828]: mon.a is new leader, mons a in quorum (ranks 0) 2023-07-14T11:52:00.309 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:00 smithi017 systemd[1]: Started Ceph mon.a for b3c386cc-223c-11ee-9b33-001a4aab830c. 2023-07-14T11:52:01.109 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout cluster: 2023-07-14T11:52:01.110 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout id: b3c386cc-223c-11ee-9b33-001a4aab830c 2023-07-14T11:52:01.110 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-07-14T11:52:01.110 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 2023-07-14T11:52:01.110 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout services: 2023-07-14T11:52:01.110 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.678791s) 2023-07-14T11:52:01.111 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-07-14T11:52:01.111 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-07-14T11:52:01.111 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 2023-07-14T11:52:01.111 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout data: 2023-07-14T11:52:01.111 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-07-14T11:52:01.111 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-07-14T11:52:01.115 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-07-14T11:52:01.116 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout pgs: 2023-07-14T11:52:01.116 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 2023-07-14T11:52:01.116 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:00.582+0000 7f871238a700 1 Processor -- start 2023-07-14T11:52:01.118 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:00.582+0000 7f871238a700 1 -- start start 2023-07-14T11:52:01.118 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:00.583+0000 7f871238a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f870c07be00 0x7f870c07a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:01.118 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:00.583+0000 7f871238a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870c07a840 con 0x7f870c07be00 2023-07-14T11:52:01.119 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:00.584+0000 7f870bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f870c07be00 0x7f870c07a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:01.119 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:00.584+0000 7f870bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f870c07be00 0x7f870c07a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46720/0 (socket says 172.21.15.17:46720) 2023-07-14T11:52:01.119 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:00.584+0000 7f870bfff700 1 -- 172.21.15.17:0/1775374238 learned_addr learned my addr 172.21.15.17:0/1775374238 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:52:01.120 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:00.584+0000 7f870bfff700 1 -- 172.21.15.17:0/1775374238 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f870c07a980 con 0x7f870c07be00 2023-07-14T11:52:01.120 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:00.584+0000 7f870bfff700 1 --2- 172.21.15.17:0/1775374238 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f870c07be00 0x7f870c07a300 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f86fc0098a0 tx=0x7f86fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a6f99cc7ff7e072 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:01.120 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:00.585+0000 7f870affd700 1 -- 172.21.15.17:0/1775374238 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f86fc004160 con 0x7f870c07be00 2023-07-14T11:52:01.120 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:00.585+0000 7f870affd700 1 -- 172.21.15.17:0/1775374238 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86fc0042c0 con 0x7f870c07be00 2023-07-14T11:52:01.120 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:00.585+0000 7f870affd700 1 -- 172.21.15.17:0/1775374238 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f86fc0045a0 con 0x7f870c07be00 2023-07-14T11:52:01.120 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:00.585+0000 7f871238a700 1 -- 172.21.15.17:0/1775374238 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f870c07be00 msgr2=0x7f870c07a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:01.120 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:00.585+0000 7f871238a700 1 --2- 172.21.15.17:0/1775374238 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f870c07be00 0x7f870c07a300 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f86fc0098a0 tx=0x7f86fc009bb0 comp rx=0 tx=0).stop 2023-07-14T11:52:01.121 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:00.585+0000 7f871238a700 1 -- 172.21.15.17:0/1775374238 shutdown_connections 2023-07-14T11:52:01.121 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:00.585+0000 7f871238a700 1 --2- 172.21.15.17:0/1775374238 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f870c07be00 0x7f870c07a300 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:01.121 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:00.585+0000 7f871238a700 1 -- 172.21.15.17:0/1775374238 >> 172.21.15.17:0/1775374238 conn(0x7f870c100970 msgr2=0x7f870c102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:01.121 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:00.585+0000 7f871238a700 1 -- 172.21.15.17:0/1775374238 shutdown_connections 2023-07-14T11:52:01.121 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:00.585+0000 7f871238a700 1 -- 172.21.15.17:0/1775374238 wait complete. 2023-07-14T11:52:01.121 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:00.587+0000 7f871238a700 1 Processor -- start 2023-07-14T11:52:01.121 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:00.587+0000 7f871238a700 1 -- start start 2023-07-14T11:52:01.122 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:00.588+0000 7f871238a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f870c07be00 0x7f870c19aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:01.122 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:00.588+0000 7f871238a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870c19afd0 con 0x7f870c07be00 2023-07-14T11:52:01.122 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:00.588+0000 7f870bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f870c07be00 0x7f870c19aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:01.122 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:00.588+0000 7f870bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f870c07be00 0x7f870c19aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46732/0 (socket says 172.21.15.17:46732) 2023-07-14T11:52:01.122 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:00.588+0000 7f870bfff700 1 -- 172.21.15.17:0/3408313553 learned_addr learned my addr 172.21.15.17:0/3408313553 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:52:01.122 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:00.588+0000 7f870bfff700 1 -- 172.21.15.17:0/3408313553 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86fc009550 con 0x7f870c07be00 2023-07-14T11:52:01.122 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:00.588+0000 7f870bfff700 1 --2- 172.21.15.17:0/3408313553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f870c07be00 0x7f870c19aa90 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f86fc004000 tx=0x7f86fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:01.122 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:00.589+0000 7f87097fa700 1 -- 172.21.15.17:0/3408313553 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f86fc011630 con 0x7f870c07be00 2023-07-14T11:52:01.123 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:00.589+0000 7f87097fa700 1 -- 172.21.15.17:0/3408313553 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86fc011790 con 0x7f870c07be00 2023-07-14T11:52:01.123 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:00.589+0000 7f87097fa700 1 -- 172.21.15.17:0/3408313553 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f86fc011a30 con 0x7f870c07be00 2023-07-14T11:52:01.123 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:00.589+0000 7f871238a700 1 -- 172.21.15.17:0/3408313553 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f870c19b1d0 con 0x7f870c07be00 2023-07-14T11:52:01.123 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:00.589+0000 7f871238a700 1 -- 172.21.15.17:0/3408313553 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f870c19b5f0 con 0x7f870c07be00 2023-07-14T11:52:01.123 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:00.589+0000 7f87097fa700 1 -- 172.21.15.17:0/3408313553 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f86fc022070 con 0x7f870c07be00 2023-07-14T11:52:01.123 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:00.589+0000 7f87097fa700 1 -- 172.21.15.17:0/3408313553 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f86fc01c450 con 0x7f870c07be00 2023-07-14T11:52:01.123 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:00.590+0000 7f871238a700 1 -- 172.21.15.17:0/3408313553 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f870c194870 con 0x7f870c07be00 2023-07-14T11:52:01.124 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:00.593+0000 7f87097fa700 1 -- 172.21.15.17:0/3408313553 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f86fc01b450 con 0x7f870c07be00 2023-07-14T11:52:01.124 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:00.669+0000 7f871238a700 1 -- 172.21.15.17:0/3408313553 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f870c02c850 con 0x7f870c07be00 2023-07-14T11:52:01.124 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:00.669+0000 7f87097fa700 1 -- 172.21.15.17:0/3408313553 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f86fc01b5f0 con 0x7f870c07be00 2023-07-14T11:52:01.124 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:00.670+0000 7f871238a700 1 -- 172.21.15.17:0/3408313553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f870c07be00 msgr2=0x7f870c19aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:01.124 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:00.670+0000 7f871238a700 1 --2- 172.21.15.17:0/3408313553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f870c07be00 0x7f870c19aa90 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f86fc004000 tx=0x7f86fc004750 comp rx=0 tx=0).stop 2023-07-14T11:52:01.124 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:00.670+0000 7f871238a700 1 -- 172.21.15.17:0/3408313553 shutdown_connections 2023-07-14T11:52:01.124 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:00.670+0000 7f871238a700 1 --2- 172.21.15.17:0/3408313553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f870c07be00 0x7f870c19aa90 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:01.124 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:00.670+0000 7f871238a700 1 -- 172.21.15.17:0/3408313553 >> 172.21.15.17:0/3408313553 conn(0x7f870c100970 msgr2=0x7f870c1022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:01.125 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:00.670+0000 7f871238a700 1 -- 172.21.15.17:0/3408313553 shutdown_connections 2023-07-14T11:52:01.125 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:00.670+0000 7f871238a700 1 -- 172.21.15.17:0/3408313553 wait complete. 2023-07-14T11:52:01.125 INFO:teuthology.orchestra.run.smithi017.stdout:mon is available 2023-07-14T11:52:01.125 INFO:teuthology.orchestra.run.smithi017.stdout:Assimilating anything we can from ceph.conf... 2023-07-14T11:52:01.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:01 smithi017 ceph-mon[125828]: mon.a is new leader, mons a in quorum (ranks 0) 2023-07-14T11:52:01.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:01 smithi017 ceph-mon[125828]: monmap e1: 1 mons at {a=[v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0]} removed_ranks: {} 2023-07-14T11:52:01.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:01 smithi017 ceph-mon[125828]: fsmap 2023-07-14T11:52:01.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:01 smithi017 ceph-mon[125828]: osdmap e1: 0 total, 0 up, 0 in 2023-07-14T11:52:01.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:01 smithi017 ceph-mon[125828]: mgrmap e1: no daemons active 2023-07-14T11:52:01.399 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:01 smithi017 ceph-mon[125828]: from='client.? 172.21.15.17:0/3408313553' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-07-14T11:52:09.620 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 2023-07-14T11:52:09.621 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout [global] 2023-07-14T11:52:09.621 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout fsid = b3c386cc-223c-11ee-9b33-001a4aab830c 2023-07-14T11:52:09.621 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.17:3300,v1:172.21.15.17:6789] 2023-07-14T11:52:09.622 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-07-14T11:52:09.622 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-07-14T11:52:09.622 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-07-14T11:52:09.623 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-07-14T11:52:09.623 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 2023-07-14T11:52:09.623 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout [mgr] 2023-07-14T11:52:09.623 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-07-14T11:52:09.623 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 2023-07-14T11:52:09.623 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout [osd] 2023-07-14T11:52:09.623 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-07-14T11:52:09.624 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-07-14T11:52:09.624 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-07-14T11:52:09.624 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-07-14T11:52:09.624 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-07-14T11:52:09.624 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:09.100+0000 7f945902b700 1 Processor -- start 2023-07-14T11:52:09.624 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:09.100+0000 7f945902b700 1 -- start start 2023-07-14T11:52:09.624 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:09.101+0000 7f945902b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f945407be00 0x7f945407a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:09.625 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:09.101+0000 7f945902b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945407a840 con 0x7f945407be00 2023-07-14T11:52:09.625 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:09.101+0000 7f9452d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f945407be00 0x7f945407a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:09.625 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:09.101+0000 7f9452d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f945407be00 0x7f945407a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46402/0 (socket says 172.21.15.17:46402) 2023-07-14T11:52:09.625 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:09.101+0000 7f9452d9d700 1 -- 172.21.15.17:0/2850945841 learned_addr learned my addr 172.21.15.17:0/2850945841 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:52:09.625 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:09.102+0000 7f9452d9d700 1 -- 172.21.15.17:0/2850945841 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f945407a980 con 0x7f945407be00 2023-07-14T11:52:09.625 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:09.102+0000 7f9452d9d700 1 --2- 172.21.15.17:0/2850945841 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f945407be00 0x7f945407a300 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f94440098a0 tx=0x7f9444009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e03d6ccbc0714d79 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:09.625 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:09.102+0000 7f9451d9b700 1 -- 172.21.15.17:0/2850945841 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9444004160 con 0x7f945407be00 2023-07-14T11:52:09.626 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:09.102+0000 7f9451d9b700 1 -- 172.21.15.17:0/2850945841 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94440042c0 con 0x7f945407be00 2023-07-14T11:52:09.626 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:09.102+0000 7f9451d9b700 1 -- 172.21.15.17:0/2850945841 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f94440045a0 con 0x7f945407be00 2023-07-14T11:52:09.626 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:09.103+0000 7f945902b700 1 -- 172.21.15.17:0/2850945841 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f945407be00 msgr2=0x7f945407a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:09.626 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:09.103+0000 7f945902b700 1 --2- 172.21.15.17:0/2850945841 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f945407be00 0x7f945407a300 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f94440098a0 tx=0x7f9444009bb0 comp rx=0 tx=0).stop 2023-07-14T11:52:09.626 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:09.103+0000 7f945902b700 1 -- 172.21.15.17:0/2850945841 shutdown_connections 2023-07-14T11:52:09.626 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:09.103+0000 7f945902b700 1 --2- 172.21.15.17:0/2850945841 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f945407be00 0x7f945407a300 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:09.626 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:09.103+0000 7f945902b700 1 -- 172.21.15.17:0/2850945841 >> 172.21.15.17:0/2850945841 conn(0x7f9454100970 msgr2=0x7f9454102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:09.626 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:09.103+0000 7f945902b700 1 -- 172.21.15.17:0/2850945841 shutdown_connections 2023-07-14T11:52:09.627 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:09.103+0000 7f945902b700 1 -- 172.21.15.17:0/2850945841 wait complete. 2023-07-14T11:52:09.627 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:09.104+0000 7f945902b700 1 Processor -- start 2023-07-14T11:52:09.627 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:09.104+0000 7f945902b700 1 -- start start 2023-07-14T11:52:09.627 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:09.104+0000 7f945902b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f945407be00 0x7f945419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:09.627 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:09.104+0000 7f945902b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945419afd0 con 0x7f945407be00 2023-07-14T11:52:09.627 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:09.105+0000 7f9452d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f945407be00 0x7f945419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:09.627 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:09.105+0000 7f9452d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f945407be00 0x7f945419aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46406/0 (socket says 172.21.15.17:46406) 2023-07-14T11:52:09.627 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:09.105+0000 7f9452d9d700 1 -- 172.21.15.17:0/2130116486 learned_addr learned my addr 172.21.15.17:0/2130116486 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:52:09.628 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:09.105+0000 7f9452d9d700 1 -- 172.21.15.17:0/2130116486 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9444009550 con 0x7f945407be00 2023-07-14T11:52:09.628 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:09.105+0000 7f9452d9d700 1 --2- 172.21.15.17:0/2130116486 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f945407be00 0x7f945419aa90 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f9444004000 tx=0x7f9444004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:09.628 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:09.105+0000 7f943bfff700 1 -- 172.21.15.17:0/2130116486 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9444011630 con 0x7f945407be00 2023-07-14T11:52:09.628 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:09.106+0000 7f943bfff700 1 -- 172.21.15.17:0/2130116486 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9444011790 con 0x7f945407be00 2023-07-14T11:52:09.628 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:09.106+0000 7f943bfff700 1 -- 172.21.15.17:0/2130116486 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9444011a30 con 0x7f945407be00 2023-07-14T11:52:09.628 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:09.106+0000 7f945902b700 1 -- 172.21.15.17:0/2130116486 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f945419b1d0 con 0x7f945407be00 2023-07-14T11:52:09.628 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:09.106+0000 7f945902b700 1 -- 172.21.15.17:0/2130116486 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f945419b5f0 con 0x7f945407be00 2023-07-14T11:52:09.629 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:09.106+0000 7f943bfff700 1 -- 172.21.15.17:0/2130116486 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9444022070 con 0x7f945407be00 2023-07-14T11:52:09.629 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:09.106+0000 7f943bfff700 1 -- 172.21.15.17:0/2130116486 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f944401c450 con 0x7f945407be00 2023-07-14T11:52:09.629 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:09.107+0000 7f945902b700 1 -- 172.21.15.17:0/2130116486 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9454194870 con 0x7f945407be00 2023-07-14T11:52:09.629 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:09.110+0000 7f943bfff700 1 -- 172.21.15.17:0/2130116486 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f944401b450 con 0x7f945407be00 2023-07-14T11:52:09.629 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:09.184+0000 7f945902b700 1 -- 172.21.15.17:0/2130116486 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f945402cad0 con 0x7f945407be00 2023-07-14T11:52:09.629 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:09.188+0000 7f943bfff700 1 -- 172.21.15.17:0/2130116486 <== mon.0 v2:172.21.15.17:3300/0 7 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f944401b5f0 con 0x7f945407be00 2023-07-14T11:52:09.629 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:09.188+0000 7f943bfff700 1 -- 172.21.15.17:0/2130116486 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f9444037630 con 0x7f945407be00 2023-07-14T11:52:09.630 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:09.189+0000 7f945902b700 1 -- 172.21.15.17:0/2130116486 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f945407be00 msgr2=0x7f945419aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:09.630 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:09.189+0000 7f945902b700 1 --2- 172.21.15.17:0/2130116486 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f945407be00 0x7f945419aa90 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f9444004000 tx=0x7f9444004750 comp rx=0 tx=0).stop 2023-07-14T11:52:09.630 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:09.189+0000 7f945902b700 1 -- 172.21.15.17:0/2130116486 shutdown_connections 2023-07-14T11:52:09.630 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:09.189+0000 7f945902b700 1 --2- 172.21.15.17:0/2130116486 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f945407be00 0x7f945419aa90 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:09.630 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:09.189+0000 7f945902b700 1 -- 172.21.15.17:0/2130116486 >> 172.21.15.17:0/2130116486 conn(0x7f9454100970 msgr2=0x7f94541022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:09.630 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:09.189+0000 7f945902b700 1 -- 172.21.15.17:0/2130116486 shutdown_connections 2023-07-14T11:52:09.630 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:09.189+0000 7f945902b700 1 -- 172.21.15.17:0/2130116486 wait complete. 2023-07-14T11:52:09.631 INFO:teuthology.orchestra.run.smithi017.stdout:Generating new minimal ceph.conf... 2023-07-14T11:52:09.631 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:09 smithi017 ceph-mon[125828]: from='client.? 172.21.15.17:0/2130116486' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-07-14T11:52:09.631 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:09 smithi017 ceph-mon[125828]: from='client.? 172.21.15.17:0/2130116486' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-07-14T11:52:10.798 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:10.201+0000 7ff6857e0700 1 Processor -- start 2023-07-14T11:52:10.798 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:10.202+0000 7ff6857e0700 1 -- start start 2023-07-14T11:52:10.799 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:10.202+0000 7ff6857e0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff68007be00 0x7ff68007a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:10.799 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:10.202+0000 7ff6857e0700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68007a840 con 0x7ff68007be00 2023-07-14T11:52:10.799 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:10.202+0000 7ff67effd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff68007be00 0x7ff68007a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:10.799 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:10.202+0000 7ff67effd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff68007be00 0x7ff68007a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46420/0 (socket says 172.21.15.17:46420) 2023-07-14T11:52:10.799 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:10.202+0000 7ff67effd700 1 -- 172.21.15.17:0/311621588 learned_addr learned my addr 172.21.15.17:0/311621588 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:52:10.799 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:10.203+0000 7ff67effd700 1 -- 172.21.15.17:0/311621588 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff68007a980 con 0x7ff68007be00 2023-07-14T11:52:10.799 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:10.203+0000 7ff67effd700 1 --2- 172.21.15.17:0/311621588 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff68007be00 0x7ff68007a300 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff6700098a0 tx=0x7ff670009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b27618b404e5b65 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:10.800 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:10.203+0000 7ff67dffb700 1 -- 172.21.15.17:0/311621588 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff670004160 con 0x7ff68007be00 2023-07-14T11:52:10.800 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:10.203+0000 7ff67dffb700 1 -- 172.21.15.17:0/311621588 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff6700042c0 con 0x7ff68007be00 2023-07-14T11:52:10.800 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:10.203+0000 7ff67dffb700 1 -- 172.21.15.17:0/311621588 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff670011a00 con 0x7ff68007be00 2023-07-14T11:52:10.800 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:10.204+0000 7ff6857e0700 1 -- 172.21.15.17:0/311621588 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff68007be00 msgr2=0x7ff68007a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:10.800 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:10.204+0000 7ff6857e0700 1 --2- 172.21.15.17:0/311621588 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff68007be00 0x7ff68007a300 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff6700098a0 tx=0x7ff670009bb0 comp rx=0 tx=0).stop 2023-07-14T11:52:10.800 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:10.204+0000 7ff6857e0700 1 -- 172.21.15.17:0/311621588 shutdown_connections 2023-07-14T11:52:10.800 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:10.204+0000 7ff6857e0700 1 --2- 172.21.15.17:0/311621588 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff68007be00 0x7ff68007a300 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:10.801 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:10.204+0000 7ff6857e0700 1 -- 172.21.15.17:0/311621588 >> 172.21.15.17:0/311621588 conn(0x7ff680100970 msgr2=0x7ff680102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:10.801 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:10.204+0000 7ff6857e0700 1 -- 172.21.15.17:0/311621588 shutdown_connections 2023-07-14T11:52:10.801 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:10.204+0000 7ff6857e0700 1 -- 172.21.15.17:0/311621588 wait complete. 2023-07-14T11:52:10.801 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:10.206+0000 7ff6857e0700 1 Processor -- start 2023-07-14T11:52:10.801 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:10.206+0000 7ff6857e0700 1 -- start start 2023-07-14T11:52:10.801 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:10.206+0000 7ff6857e0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff68007be00 0x7ff68019b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:10.801 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:10.206+0000 7ff6857e0700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68019b880 con 0x7ff68007be00 2023-07-14T11:52:10.802 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:10.207+0000 7ff67effd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff68007be00 0x7ff68019b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:10.802 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:10.207+0000 7ff67effd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff68007be00 0x7ff68019b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46426/0 (socket says 172.21.15.17:46426) 2023-07-14T11:52:10.802 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:10.207+0000 7ff67effd700 1 -- 172.21.15.17:0/1868388014 learned_addr learned my addr 172.21.15.17:0/1868388014 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:52:10.802 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:10.207+0000 7ff67effd700 1 -- 172.21.15.17:0/1868388014 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff670009550 con 0x7ff68007be00 2023-07-14T11:52:10.802 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:10.207+0000 7ff67effd700 1 --2- 172.21.15.17:0/1868388014 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff68007be00 0x7ff68019b340 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff670009130 tx=0x7ff670011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:10.802 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:10.207+0000 7ff667fff700 1 -- 172.21.15.17:0/1868388014 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff670003ab0 con 0x7ff68007be00 2023-07-14T11:52:10.802 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:10.207+0000 7ff667fff700 1 -- 172.21.15.17:0/1868388014 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff670003c10 con 0x7ff68007be00 2023-07-14T11:52:10.802 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:10.207+0000 7ff667fff700 1 -- 172.21.15.17:0/1868388014 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff6700247d0 con 0x7ff68007be00 2023-07-14T11:52:10.803 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:10.207+0000 7ff6857e0700 1 -- 172.21.15.17:0/1868388014 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff68019ba80 con 0x7ff68007be00 2023-07-14T11:52:10.803 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:10.208+0000 7ff6857e0700 1 -- 172.21.15.17:0/1868388014 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff68019bea0 con 0x7ff68007be00 2023-07-14T11:52:10.803 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:10.208+0000 7ff667fff700 1 -- 172.21.15.17:0/1868388014 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff670024d00 con 0x7ff68007be00 2023-07-14T11:52:10.803 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:10.208+0000 7ff667fff700 1 -- 172.21.15.17:0/1868388014 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff6700193c0 con 0x7ff68007be00 2023-07-14T11:52:10.803 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:10.209+0000 7ff6857e0700 1 -- 172.21.15.17:0/1868388014 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6801952f0 con 0x7ff68007be00 2023-07-14T11:52:10.803 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:10.211+0000 7ff667fff700 1 -- 172.21.15.17:0/1868388014 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7ff67001abb0 con 0x7ff68007be00 2023-07-14T11:52:10.803 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:10.284+0000 7ff6857e0700 1 -- 172.21.15.17:0/1868388014 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7ff68002c850 con 0x7ff68007be00 2023-07-14T11:52:10.803 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:10.285+0000 7ff667fff700 1 -- 172.21.15.17:0/1868388014 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7ff67001ad50 con 0x7ff68007be00 2023-07-14T11:52:10.804 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:10.286+0000 7ff6857e0700 1 -- 172.21.15.17:0/1868388014 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff68007be00 msgr2=0x7ff68019b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:10.804 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:10.286+0000 7ff6857e0700 1 --2- 172.21.15.17:0/1868388014 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff68007be00 0x7ff68019b340 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff670009130 tx=0x7ff670011ad0 comp rx=0 tx=0).stop 2023-07-14T11:52:10.804 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:10.286+0000 7ff6857e0700 1 -- 172.21.15.17:0/1868388014 shutdown_connections 2023-07-14T11:52:10.804 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:10.286+0000 7ff6857e0700 1 --2- 172.21.15.17:0/1868388014 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff68007be00 0x7ff68019b340 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:10.804 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:10.286+0000 7ff6857e0700 1 -- 172.21.15.17:0/1868388014 >> 172.21.15.17:0/1868388014 conn(0x7ff680100970 msgr2=0x7ff6801020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:10.804 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:10.286+0000 7ff6857e0700 1 -- 172.21.15.17:0/1868388014 shutdown_connections 2023-07-14T11:52:10.804 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:10.286+0000 7ff6857e0700 1 -- 172.21.15.17:0/1868388014 wait complete. 2023-07-14T11:52:10.805 INFO:teuthology.orchestra.run.smithi017.stdout:Restarting the monitor... 2023-07-14T11:52:11.148 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:10 smithi017 systemd[1]: Stopping Ceph mon.a for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T11:52:11.148 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:10 smithi017 ceph-mon[125828]: from='client.? 172.21.15.17:0/1868388014' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:52:11.589 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:11 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mon-a[125804]: 2023-07-14T11:52:11.316+0000 7f2ae1753700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T11:52:11.589 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:11 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mon-a[125804]: 2023-07-14T11:52:11.316+0000 7f2ae1753700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-07-14T11:52:11.864 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:11 smithi017 bash[126167]: ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mon-a 2023-07-14T11:52:12.148 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 systemd[1]: ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mon.a.service: Succeeded. 2023-07-14T11:52:12.148 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 systemd[1]: Stopped Ceph mon.a for b3c386cc-223c-11ee-9b33-001a4aab830c. 2023-07-14T11:52:12.149 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 systemd[1]: Starting Ceph mon.a for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T11:52:12.837 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 podman[126299]: 2023-07-14T11:52:12.837 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 bash[126299]: a436742b0c29002bc49dbd8a24ef46c0f1e7f87e06671789a55341ccce706a2b 2023-07-14T11:52:12.838 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: set uid:gid to 167:167 (ceph:ceph) 2023-07-14T11:52:12.855 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev), process ceph-mon, pid 7 2023-07-14T11:52:12.855 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: pidfile_write: ignore empty --pid-file 2023-07-14T11:52:12.855 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: load: jerasure load: lrc 2023-07-14T11:52:12.855 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: RocksDB version: 7.9.2 2023-07-14T11:52:12.856 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Git sha 0 2023-07-14T11:52:12.856 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Compile date 2023-07-13 18:29:45 2023-07-14T11:52:12.856 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: DB SUMMARY 2023-07-14T11:52:12.856 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: DB Session ID: 7NSTSPSWE8DG61JEYVNC 2023-07-14T11:52:12.856 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: CURRENT file: CURRENT 2023-07-14T11:52:12.856 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: IDENTITY file: IDENTITY 2023-07-14T11:52:12.856 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-07-14T11:52:12.857 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-07-14T11:52:12.857 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 99747 ; 2023-07-14T11:52:12.857 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.error_if_exists: 0 2023-07-14T11:52:12.857 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.create_if_missing: 0 2023-07-14T11:52:12.857 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.paranoid_checks: 1 2023-07-14T11:52:12.857 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.flush_verify_memtable_count: 1 2023-07-14T11:52:12.857 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-07-14T11:52:12.858 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-07-14T11:52:12.858 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.env: 0x55ca1a45e720 2023-07-14T11:52:12.858 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.fs: PosixFileSystem 2023-07-14T11:52:12.858 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.info_log: 0x55ca1b17f280 2023-07-14T11:52:12.858 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.max_file_opening_threads: 16 2023-07-14T11:52:12.858 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.statistics: (nil) 2023-07-14T11:52:12.858 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.use_fsync: 0 2023-07-14T11:52:12.858 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.max_log_file_size: 0 2023-07-14T11:52:12.859 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-07-14T11:52:12.859 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.log_file_time_to_roll: 0 2023-07-14T11:52:12.859 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.keep_log_file_num: 1000 2023-07-14T11:52:12.859 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.recycle_log_file_num: 0 2023-07-14T11:52:12.859 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.allow_fallocate: 1 2023-07-14T11:52:12.859 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.allow_mmap_reads: 0 2023-07-14T11:52:12.859 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.allow_mmap_writes: 0 2023-07-14T11:52:12.859 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.use_direct_reads: 0 2023-07-14T11:52:12.860 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-07-14T11:52:12.860 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.create_missing_column_families: 0 2023-07-14T11:52:12.860 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.db_log_dir: 2023-07-14T11:52:12.860 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.wal_dir: 2023-07-14T11:52:12.860 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.table_cache_numshardbits: 6 2023-07-14T11:52:12.861 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.WAL_ttl_seconds: 0 2023-07-14T11:52:12.861 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.WAL_size_limit_MB: 0 2023-07-14T11:52:12.862 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-07-14T11:52:12.862 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-07-14T11:52:12.862 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.is_fd_close_on_exec: 1 2023-07-14T11:52:12.862 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.advise_random_on_open: 1 2023-07-14T11:52:12.862 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.db_write_buffer_size: 0 2023-07-14T11:52:12.862 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.write_buffer_manager: 0x55ca1b0a03c0 2023-07-14T11:52:12.862 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-07-14T11:52:12.862 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-07-14T11:52:12.863 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.use_adaptive_mutex: 0 2023-07-14T11:52:12.863 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.rate_limiter: (nil) 2023-07-14T11:52:12.863 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-07-14T11:52:12.863 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.wal_recovery_mode: 2 2023-07-14T11:52:12.863 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.enable_thread_tracking: 0 2023-07-14T11:52:12.863 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.enable_pipelined_write: 0 2023-07-14T11:52:12.863 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.unordered_write: 0 2023-07-14T11:52:12.863 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-07-14T11:52:12.864 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-07-14T11:52:12.864 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-07-14T11:52:12.864 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-07-14T11:52:12.864 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.row_cache: None 2023-07-14T11:52:12.864 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.wal_filter: None 2023-07-14T11:52:12.864 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-07-14T11:52:12.865 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.allow_ingest_behind: 0 2023-07-14T11:52:12.865 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.two_write_queues: 0 2023-07-14T11:52:12.865 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.manual_wal_flush: 0 2023-07-14T11:52:12.865 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.wal_compression: 0 2023-07-14T11:52:12.865 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.atomic_flush: 0 2023-07-14T11:52:12.865 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-07-14T11:52:12.865 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.persist_stats_to_disk: 0 2023-07-14T11:52:12.866 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.write_dbid_to_manifest: 0 2023-07-14T11:52:12.866 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.log_readahead_size: 0 2023-07-14T11:52:12.866 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-07-14T11:52:12.866 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.best_efforts_recovery: 0 2023-07-14T11:52:12.866 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-07-14T11:52:12.866 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-07-14T11:52:12.866 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.allow_data_in_errors: 0 2023-07-14T11:52:12.867 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.db_host_id: __hostname__ 2023-07-14T11:52:12.867 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.enforce_single_del_contracts: true 2023-07-14T11:52:12.867 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.max_background_jobs: 2 2023-07-14T11:52:12.867 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.max_background_compactions: -1 2023-07-14T11:52:12.867 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.max_subcompactions: 1 2023-07-14T11:52:12.867 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-07-14T11:52:12.867 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-07-14T11:52:12.867 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.delayed_write_rate : 16777216 2023-07-14T11:52:12.868 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.max_total_wal_size: 0 2023-07-14T11:52:12.868 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-07-14T11:52:12.868 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.stats_dump_period_sec: 600 2023-07-14T11:52:12.868 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.stats_persist_period_sec: 600 2023-07-14T11:52:12.868 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-07-14T11:52:12.868 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.max_open_files: -1 2023-07-14T11:52:12.868 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.bytes_per_sync: 0 2023-07-14T11:52:12.869 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.wal_bytes_per_sync: 0 2023-07-14T11:52:12.869 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.strict_bytes_per_sync: 0 2023-07-14T11:52:12.869 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.compaction_readahead_size: 0 2023-07-14T11:52:12.869 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.max_background_flushes: -1 2023-07-14T11:52:12.869 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Compression algorithms supported: 2023-07-14T11:52:12.869 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-07-14T11:52:12.869 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: kZSTD supported: 0 2023-07-14T11:52:12.869 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: kXpressCompression supported: 0 2023-07-14T11:52:12.870 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: kLZ4HCCompression supported: 1 2023-07-14T11:52:12.870 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: kZlibCompression supported: 1 2023-07-14T11:52:12.870 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: kSnappyCompression supported: 1 2023-07-14T11:52:12.870 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: kLZ4Compression supported: 1 2023-07-14T11:52:12.870 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: kBZip2Compression supported: 0 2023-07-14T11:52:12.870 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Fast CRC32 supported: Supported on x86 2023-07-14T11:52:12.870 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: DMutex implementation: pthread_mutex_t 2023-07-14T11:52:12.870 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-07-14T11:52:12.871 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-07-14T11:52:12.871 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-07-14T11:52:12.871 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.merge_operator: 2023-07-14T11:52:12.871 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.compaction_filter: None 2023-07-14T11:52:12.871 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.compaction_filter_factory: None 2023-07-14T11:52:12.871 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.sst_partitioner_factory: None 2023-07-14T11:52:12.871 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.memtable_factory: SkipListFactory 2023-07-14T11:52:12.871 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.table_factory: BlockBasedTable 2023-07-14T11:52:12.872 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ca1b17f380) 2023-07-14T11:52:12.872 INFO:journalctl@ceph.mon.a.smithi017.stdout: cache_index_and_filter_blocks: 1 2023-07-14T11:52:12.872 INFO:journalctl@ceph.mon.a.smithi017.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-07-14T11:52:12.872 INFO:journalctl@ceph.mon.a.smithi017.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-07-14T11:52:12.872 INFO:journalctl@ceph.mon.a.smithi017.stdout: pin_top_level_index_and_filter: 1 2023-07-14T11:52:12.872 INFO:journalctl@ceph.mon.a.smithi017.stdout: index_type: 0 2023-07-14T11:52:12.872 INFO:journalctl@ceph.mon.a.smithi017.stdout: data_block_index_type: 0 2023-07-14T11:52:12.873 INFO:journalctl@ceph.mon.a.smithi017.stdout: index_shortening: 1 2023-07-14T11:52:12.873 INFO:journalctl@ceph.mon.a.smithi017.stdout: data_block_hash_table_util_ratio: 0.750000 2023-07-14T11:52:12.873 INFO:journalctl@ceph.mon.a.smithi017.stdout: checksum: 4 2023-07-14T11:52:12.873 INFO:journalctl@ceph.mon.a.smithi017.stdout: no_block_cache: 0 2023-07-14T11:52:12.873 INFO:journalctl@ceph.mon.a.smithi017.stdout: block_cache: 0x55ca1b13f1f0 2023-07-14T11:52:12.873 INFO:journalctl@ceph.mon.a.smithi017.stdout: block_cache_name: BinnedLRUCache 2023-07-14T11:52:12.873 INFO:journalctl@ceph.mon.a.smithi017.stdout: block_cache_options: 2023-07-14T11:52:12.873 INFO:journalctl@ceph.mon.a.smithi017.stdout: capacity : 536870912 2023-07-14T11:52:12.874 INFO:journalctl@ceph.mon.a.smithi017.stdout: num_shard_bits : 4 2023-07-14T11:52:12.874 INFO:journalctl@ceph.mon.a.smithi017.stdout: strict_capacity_limit : 0 2023-07-14T11:52:12.874 INFO:journalctl@ceph.mon.a.smithi017.stdout: high_pri_pool_ratio: 0.000 2023-07-14T11:52:12.874 INFO:journalctl@ceph.mon.a.smithi017.stdout: block_cache_compressed: (nil) 2023-07-14T11:52:12.874 INFO:journalctl@ceph.mon.a.smithi017.stdout: persistent_cache: (nil) 2023-07-14T11:52:12.874 INFO:journalctl@ceph.mon.a.smithi017.stdout: block_size: 4096 2023-07-14T11:52:12.874 INFO:journalctl@ceph.mon.a.smithi017.stdout: block_size_deviation: 10 2023-07-14T11:52:12.874 INFO:journalctl@ceph.mon.a.smithi017.stdout: block_restart_interval: 16 2023-07-14T11:52:12.875 INFO:journalctl@ceph.mon.a.smithi017.stdout: index_block_restart_interval: 1 2023-07-14T11:52:12.875 INFO:journalctl@ceph.mon.a.smithi017.stdout: metadata_block_size: 4096 2023-07-14T11:52:12.875 INFO:journalctl@ceph.mon.a.smithi017.stdout: partition_filters: 0 2023-07-14T11:52:12.875 INFO:journalctl@ceph.mon.a.smithi017.stdout: use_delta_encoding: 1 2023-07-14T11:52:12.875 INFO:journalctl@ceph.mon.a.smithi017.stdout: filter_policy: bloomfilter 2023-07-14T11:52:12.875 INFO:journalctl@ceph.mon.a.smithi017.stdout: whole_key_filtering: 1 2023-07-14T11:52:12.876 INFO:journalctl@ceph.mon.a.smithi017.stdout: verify_compression: 0 2023-07-14T11:52:12.876 INFO:journalctl@ceph.mon.a.smithi017.stdout: read_amp_bytes_per_bit: 0 2023-07-14T11:52:12.876 INFO:journalctl@ceph.mon.a.smithi017.stdout: format_version: 5 2023-07-14T11:52:12.876 INFO:journalctl@ceph.mon.a.smithi017.stdout: enable_index_compression: 1 2023-07-14T11:52:12.876 INFO:journalctl@ceph.mon.a.smithi017.stdout: block_align: 0 2023-07-14T11:52:12.876 INFO:journalctl@ceph.mon.a.smithi017.stdout: max_auto_readahead_size: 262144 2023-07-14T11:52:12.876 INFO:journalctl@ceph.mon.a.smithi017.stdout: prepopulate_block_cache: 0 2023-07-14T11:52:12.877 INFO:journalctl@ceph.mon.a.smithi017.stdout: initial_auto_readahead_size: 8192 2023-07-14T11:52:12.877 INFO:journalctl@ceph.mon.a.smithi017.stdout: num_file_reads_for_auto_readahead: 2 2023-07-14T11:52:12.877 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.write_buffer_size: 33554432 2023-07-14T11:52:12.877 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.max_write_buffer_number: 2 2023-07-14T11:52:12.877 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.compression: NoCompression 2023-07-14T11:52:12.877 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.bottommost_compression: Disabled 2023-07-14T11:52:12.877 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.prefix_extractor: nullptr 2023-07-14T11:52:12.877 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-07-14T11:52:12.878 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.num_levels: 7 2023-07-14T11:52:12.878 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-07-14T11:52:12.878 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-07-14T11:52:12.878 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-07-14T11:52:12.878 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-07-14T11:52:12.878 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-07-14T11:52:12.878 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-07-14T11:52:12.878 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-07-14T11:52:12.879 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-07-14T11:52:12.879 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-07-14T11:52:12.879 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-07-14T11:52:12.879 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-07-14T11:52:12.879 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-07-14T11:52:12.879 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.compression_opts.window_bits: -14 2023-07-14T11:52:12.879 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.compression_opts.level: 32767 2023-07-14T11:52:12.880 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.compression_opts.strategy: 0 2023-07-14T11:52:12.880 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-07-14T11:52:12.880 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-07-14T11:52:12.880 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-07-14T11:52:12.880 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-07-14T11:52:12.880 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.compression_opts.enabled: false 2023-07-14T11:52:12.880 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-07-14T11:52:12.880 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-07-14T11:52:12.881 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-07-14T11:52:12.881 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-07-14T11:52:12.881 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.target_file_size_base: 67108864 2023-07-14T11:52:12.881 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.target_file_size_multiplier: 1 2023-07-14T11:52:12.881 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-07-14T11:52:12.881 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-07-14T11:52:12.881 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-07-14T11:52:12.881 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-07-14T11:52:12.882 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-07-14T11:52:12.882 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-07-14T11:52:12.882 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-07-14T11:52:12.882 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-07-14T11:52:12.882 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-07-14T11:52:12.882 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-07-14T11:52:12.882 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-07-14T11:52:12.882 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-07-14T11:52:12.883 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-07-14T11:52:12.883 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.arena_block_size: 1048576 2023-07-14T11:52:12.883 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-07-14T11:52:12.883 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-07-14T11:52:12.883 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.disable_auto_compactions: 0 2023-07-14T11:52:12.883 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-07-14T11:52:12.883 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-07-14T11:52:12.884 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-07-14T11:52:12.884 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-07-14T11:52:12.884 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-07-14T11:52:12.884 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-07-14T11:52:12.884 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-07-14T11:52:12.884 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-07-14T11:52:12.884 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-07-14T11:52:12.885 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-07-14T11:52:12.885 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.table_properties_collectors: 2023-07-14T11:52:12.885 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.inplace_update_support: 0 2023-07-14T11:52:12.885 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.inplace_update_num_locks: 10000 2023-07-14T11:52:12.885 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-07-14T11:52:12.885 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-07-14T11:52:12.885 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.memtable_huge_page_size: 0 2023-07-14T11:52:12.886 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.bloom_locality: 0 2023-07-14T11:52:12.886 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.max_successive_merges: 0 2023-07-14T11:52:12.886 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.optimize_filters_for_hits: 0 2023-07-14T11:52:12.886 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.paranoid_file_checks: 0 2023-07-14T11:52:12.886 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.force_consistency_checks: 1 2023-07-14T11:52:12.886 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.report_bg_io_stats: 0 2023-07-14T11:52:12.886 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.ttl: 2592000 2023-07-14T11:52:12.887 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.periodic_compaction_seconds: 0 2023-07-14T11:52:12.887 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-07-14T11:52:12.887 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-07-14T11:52:12.887 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.enable_blob_files: false 2023-07-14T11:52:12.887 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.min_blob_size: 0 2023-07-14T11:52:12.887 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.blob_file_size: 268435456 2023-07-14T11:52:12.887 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.blob_compression_type: NoCompression 2023-07-14T11:52:12.888 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.enable_blob_garbage_collection: false 2023-07-14T11:52:12.888 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-07-14T11:52:12.888 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-07-14T11:52:12.888 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-07-14T11:52:12.888 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.blob_file_starting_level: 0 2023-07-14T11:52:12.888 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-07-14T11:52:12.888 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-07-14T11:52:12.888 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-07-14T11:52:12.889 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 1b673fd7-404c-4532-b95b-b1398292df9e 2023-07-14T11:52:12.889 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689335532863039, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-07-14T11:52:12.889 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-07-14T11:52:12.889 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689335532864806, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 94500, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 332, "table_properties": {"data_size": 92505, "index_size": 277, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13672, "raw_average_key_size": 49, "raw_value_size": 85044, "raw_average_value_size": 309, "num_data_blocks": 11, "num_entries": 275, "num_filter_entries": 275, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1689335532, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "1b673fd7-404c-4532-b95b-b1398292df9e", "db_session_id": "7NSTSPSWE8DG61JEYVNC", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-07-14T11:52:12.889 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689335532864941, "job": 1, "event": "recovery_finished"} 2023-07-14T11:52:12.889 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-07-14T11:52:12.889 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-07-14T11:52:12.890 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ca1b1d8000 2023-07-14T11:52:12.890 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: DB pointer 0x55ca1b1c4000 2023-07-14T11:52:12.890 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-07-14T11:52:12.890 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-07-14T11:52:12.890 INFO:journalctl@ceph.mon.a.smithi017.stdout: ** DB Stats ** 2023-07-14T11:52:12.890 INFO:journalctl@ceph.mon.a.smithi017.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-14T11:52:12.890 INFO:journalctl@ceph.mon.a.smithi017.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-07-14T11:52:12.890 INFO:journalctl@ceph.mon.a.smithi017.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-14T11:52:12.891 INFO:journalctl@ceph.mon.a.smithi017.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-14T11:52:12.891 INFO:journalctl@ceph.mon.a.smithi017.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-07-14T11:52:12.891 INFO:journalctl@ceph.mon.a.smithi017.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-14T11:52:12.891 INFO:journalctl@ceph.mon.a.smithi017.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-14T11:52:12.891 INFO:journalctl@ceph.mon.a.smithi017.stdout: 2023-07-14T11:52:12.891 INFO:journalctl@ceph.mon.a.smithi017.stdout: ** Compaction Stats [default] ** 2023-07-14T11:52:12.892 INFO:journalctl@ceph.mon.a.smithi017.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-14T11:52:12.892 INFO:journalctl@ceph.mon.a.smithi017.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-07-14T11:52:12.892 INFO:journalctl@ceph.mon.a.smithi017.stdout: L0 2/0 94.12 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 80.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-14T11:52:12.892 INFO:journalctl@ceph.mon.a.smithi017.stdout: Sum 2/0 94.12 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 80.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-14T11:52:12.892 INFO:journalctl@ceph.mon.a.smithi017.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 80.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-14T11:52:12.892 INFO:journalctl@ceph.mon.a.smithi017.stdout: 2023-07-14T11:52:12.892 INFO:journalctl@ceph.mon.a.smithi017.stdout: ** Compaction Stats [default] ** 2023-07-14T11:52:12.893 INFO:journalctl@ceph.mon.a.smithi017.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-14T11:52:12.893 INFO:journalctl@ceph.mon.a.smithi017.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-07-14T11:52:12.893 INFO:journalctl@ceph.mon.a.smithi017.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 80.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-14T11:52:12.893 INFO:journalctl@ceph.mon.a.smithi017.stdout: 2023-07-14T11:52:12.893 INFO:journalctl@ceph.mon.a.smithi017.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-07-14T11:52:12.893 INFO:journalctl@ceph.mon.a.smithi017.stdout: 2023-07-14T11:52:12.893 INFO:journalctl@ceph.mon.a.smithi017.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-14T11:52:12.893 INFO:journalctl@ceph.mon.a.smithi017.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-07-14T11:52:12.894 INFO:journalctl@ceph.mon.a.smithi017.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-07-14T11:52:12.894 INFO:journalctl@ceph.mon.a.smithi017.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-07-14T11:52:12.894 INFO:journalctl@ceph.mon.a.smithi017.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-07-14T11:52:12.894 INFO:journalctl@ceph.mon.a.smithi017.stdout: AddFile(Keys): cumulative 0, interval 0 2023-07-14T11:52:12.894 INFO:journalctl@ceph.mon.a.smithi017.stdout: Cumulative compaction: 0.00 GB write, 14.45 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-14T11:52:12.894 INFO:journalctl@ceph.mon.a.smithi017.stdout: Interval compaction: 0.00 GB write, 14.45 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-14T11:52:12.894 INFO:journalctl@ceph.mon.a.smithi017.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-07-14T11:52:12.895 INFO:journalctl@ceph.mon.a.smithi017.stdout: Block cache BinnedLRUCache@0x55ca1b13f1f0#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2023-07-14T11:52:12.895 INFO:journalctl@ceph.mon.a.smithi017.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2023-07-14T11:52:12.895 INFO:journalctl@ceph.mon.a.smithi017.stdout: 2023-07-14T11:52:12.895 INFO:journalctl@ceph.mon.a.smithi017.stdout: ** File Read Latency Histogram By Level [default] ** 2023-07-14T11:52:12.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: starting mon.a rank 0 at public addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] at bind addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid b3c386cc-223c-11ee-9b33-001a4aab830c 2023-07-14T11:52:12.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: mon.a@-1(???) e1 preinit fsid b3c386cc-223c-11ee-9b33-001a4aab830c 2023-07-14T11:52:12.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: mon.a@-1(???).mds e1 new map 2023-07-14T11:52:12.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: mon.a@-1(???).mds e1 print_map 2023-07-14T11:52:12.896 INFO:journalctl@ceph.mon.a.smithi017.stdout: e1 2023-07-14T11:52:12.896 INFO:journalctl@ceph.mon.a.smithi017.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-07-14T11:52:12.896 INFO:journalctl@ceph.mon.a.smithi017.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-14T11:52:12.896 INFO:journalctl@ceph.mon.a.smithi017.stdout: legacy client fscid: -1 2023-07-14T11:52:12.896 INFO:journalctl@ceph.mon.a.smithi017.stdout: 2023-07-14T11:52:12.897 INFO:journalctl@ceph.mon.a.smithi017.stdout: No filesystems configured 2023-07-14T11:52:12.897 INFO:teuthology.orchestra.run.smithi017.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2023-07-14T11:52:13.120 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-07-14T11:52:13.120 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-07-14T11:52:13.121 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-07-14T11:52:13.121 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-07-14T11:52:13.121 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-07-14T11:52:13.122 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: expand_channel_meta expand map: {default=false} 2023-07-14T11:52:13.122 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: expand_channel_meta from 'false' to 'false' 2023-07-14T11:52:13.122 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: expand_channel_meta expanded map: {default=false} 2023-07-14T11:52:13.122 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: expand_channel_meta expand map: {default=info} 2023-07-14T11:52:13.123 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: expand_channel_meta from 'info' to 'info' 2023-07-14T11:52:13.123 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: expand_channel_meta expanded map: {default=info} 2023-07-14T11:52:13.123 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: expand_channel_meta expand map: {default=daemon} 2023-07-14T11:52:13.124 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: expand_channel_meta from 'daemon' to 'daemon' 2023-07-14T11:52:13.124 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: expand_channel_meta expanded map: {default=daemon} 2023-07-14T11:52:13.124 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: expand_channel_meta expand map: {default=debug} 2023-07-14T11:52:13.124 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: expand_channel_meta from 'debug' to 'debug' 2023-07-14T11:52:13.125 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: expand_channel_meta expanded map: {default=debug} 2023-07-14T11:52:13.125 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 systemd[1]: Started Ceph mon.a for b3c386cc-223c-11ee-9b33-001a4aab830c. 2023-07-14T11:52:13.125 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: mon.a is new leader, mons a in quorum (ranks 0) 2023-07-14T11:52:13.125 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: monmap e1: 1 mons at {a=[v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0]} removed_ranks: {} 2023-07-14T11:52:13.126 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: fsmap 2023-07-14T11:52:13.126 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: osdmap e1: 0 total, 0 up, 0 in 2023-07-14T11:52:13.126 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:12 smithi017 ceph-mon[126339]: mgrmap e1: no daemons active 2023-07-14T11:52:13.952 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:13.403+0000 7f908e46f700 1 Processor -- start 2023-07-14T11:52:13.952 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:13.403+0000 7f908e46f700 1 -- start start 2023-07-14T11:52:13.952 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:13.403+0000 7f908e46f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f908807bde0 0x7f908807a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:13.953 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:13.403+0000 7f908e46f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908807a820 con 0x7f908807bde0 2023-07-14T11:52:13.953 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:13.404+0000 7f908d46d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f908807bde0 0x7f908807a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:13.957 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:13.404+0000 7f908d46d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f908807bde0 0x7f908807a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41134/0 (socket says 172.21.15.17:41134) 2023-07-14T11:52:13.957 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:13.404+0000 7f908d46d700 1 -- 172.21.15.17:0/1951252101 learned_addr learned my addr 172.21.15.17:0/1951252101 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:52:13.957 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:13.404+0000 7f908d46d700 1 -- 172.21.15.17:0/1951252101 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f908807a960 con 0x7f908807bde0 2023-07-14T11:52:13.958 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:13.404+0000 7f908d46d700 1 --2- 172.21.15.17:0/1951252101 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f908807bde0 0x7f908807a2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f90840098a0 tx=0x7f9084009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff2bb6fd217e42b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:13.958 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:13.405+0000 7f907ffff700 1 -- 172.21.15.17:0/1951252101 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9084004160 con 0x7f908807bde0 2023-07-14T11:52:13.958 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:13.405+0000 7f907ffff700 1 -- 172.21.15.17:0/1951252101 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f90840042c0 con 0x7f908807bde0 2023-07-14T11:52:13.958 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:13.405+0000 7f907ffff700 1 -- 172.21.15.17:0/1951252101 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9084011a00 con 0x7f908807bde0 2023-07-14T11:52:13.958 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:13.405+0000 7f908e46f700 1 -- 172.21.15.17:0/1951252101 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f908807bde0 msgr2=0x7f908807a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:13.958 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:13.405+0000 7f908e46f700 1 --2- 172.21.15.17:0/1951252101 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f908807bde0 0x7f908807a2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f90840098a0 tx=0x7f9084009bb0 comp rx=0 tx=0).stop 2023-07-14T11:52:13.958 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:13.405+0000 7f908e46f700 1 -- 172.21.15.17:0/1951252101 shutdown_connections 2023-07-14T11:52:13.959 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:13.405+0000 7f908e46f700 1 --2- 172.21.15.17:0/1951252101 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f908807bde0 0x7f908807a2e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:13.959 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:13.405+0000 7f908e46f700 1 -- 172.21.15.17:0/1951252101 >> 172.21.15.17:0/1951252101 conn(0x7f90881007f0 msgr2=0x7f9088102c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:13.959 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:13.406+0000 7f908e46f700 1 -- 172.21.15.17:0/1951252101 shutdown_connections 2023-07-14T11:52:13.959 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:13.406+0000 7f908e46f700 1 -- 172.21.15.17:0/1951252101 wait complete. 2023-07-14T11:52:13.959 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:13.407+0000 7f908e46f700 1 Processor -- start 2023-07-14T11:52:13.959 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:13.407+0000 7f908e46f700 1 -- start start 2023-07-14T11:52:13.959 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:13.407+0000 7f908e46f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f908807bde0 0x7f908819b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:13.960 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:13.407+0000 7f908e46f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908819b630 con 0x7f908807bde0 2023-07-14T11:52:13.960 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:13.407+0000 7f908d46d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f908807bde0 0x7f908819b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:13.960 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:13.407+0000 7f908d46d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f908807bde0 0x7f908819b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41140/0 (socket says 172.21.15.17:41140) 2023-07-14T11:52:13.960 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:13.407+0000 7f908d46d700 1 -- 172.21.15.17:0/4184650799 learned_addr learned my addr 172.21.15.17:0/4184650799 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:52:13.960 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:13.408+0000 7f908d46d700 1 -- 172.21.15.17:0/4184650799 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9084009550 con 0x7f908807bde0 2023-07-14T11:52:13.960 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:13.408+0000 7f908d46d700 1 --2- 172.21.15.17:0/4184650799 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f908807bde0 0x7f908819b0f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9084006af0 tx=0x7f9084011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:13.961 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:13.408+0000 7f907e7fc700 1 -- 172.21.15.17:0/4184650799 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9084003a40 con 0x7f908807bde0 2023-07-14T11:52:13.961 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:13.408+0000 7f907e7fc700 1 -- 172.21.15.17:0/4184650799 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9084003ba0 con 0x7f908807bde0 2023-07-14T11:52:13.961 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:13.408+0000 7f908e46f700 1 -- 172.21.15.17:0/4184650799 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f908819b830 con 0x7f908807bde0 2023-07-14T11:52:13.961 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:13.408+0000 7f907e7fc700 1 -- 172.21.15.17:0/4184650799 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f908401a700 con 0x7f908807bde0 2023-07-14T11:52:13.961 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:13.408+0000 7f908e46f700 1 -- 172.21.15.17:0/4184650799 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f908819bc50 con 0x7f908807bde0 2023-07-14T11:52:13.961 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:13.409+0000 7f907e7fc700 1 -- 172.21.15.17:0/4184650799 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f908401ac80 con 0x7f908807bde0 2023-07-14T11:52:13.961 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:13.409+0000 7f907e7fc700 1 -- 172.21.15.17:0/4184650799 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9084019700 con 0x7f908807bde0 2023-07-14T11:52:13.962 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:13.410+0000 7f908e46f700 1 -- 172.21.15.17:0/4184650799 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9088195110 con 0x7f908807bde0 2023-07-14T11:52:13.962 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:13.413+0000 7f907e7fc700 1 -- 172.21.15.17:0/4184650799 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f9084024ad0 con 0x7f908807bde0 2023-07-14T11:52:13.962 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:13.485+0000 7f908e46f700 1 -- 172.21.15.17:0/4184650799 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f908802ca50 con 0x7f908807bde0 2023-07-14T11:52:13.962 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:13.488+0000 7f907e7fc700 1 -- 172.21.15.17:0/4184650799 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f9084024c70 con 0x7f908807bde0 2023-07-14T11:52:13.962 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:13.488+0000 7f908e46f700 1 -- 172.21.15.17:0/4184650799 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f908807bde0 msgr2=0x7f908819b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:13.962 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:13.488+0000 7f908e46f700 1 --2- 172.21.15.17:0/4184650799 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f908807bde0 0x7f908819b0f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9084006af0 tx=0x7f9084011ad0 comp rx=0 tx=0).stop 2023-07-14T11:52:13.962 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:13.489+0000 7f908e46f700 1 -- 172.21.15.17:0/4184650799 shutdown_connections 2023-07-14T11:52:13.963 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:13.489+0000 7f908e46f700 1 --2- 172.21.15.17:0/4184650799 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f908807bde0 0x7f908819b0f0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:13.963 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:13.489+0000 7f908e46f700 1 -- 172.21.15.17:0/4184650799 >> 172.21.15.17:0/4184650799 conn(0x7f90881007f0 msgr2=0x7f9088101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:13.963 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:13.489+0000 7f908e46f700 1 -- 172.21.15.17:0/4184650799 shutdown_connections 2023-07-14T11:52:13.963 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:13.489+0000 7f908e46f700 1 -- 172.21.15.17:0/4184650799 wait complete. 2023-07-14T11:52:13.963 INFO:teuthology.orchestra.run.smithi017.stdout:Wrote config to /etc/ceph/ceph.conf 2023-07-14T11:52:13.983 INFO:teuthology.orchestra.run.smithi017.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-07-14T11:52:13.983 INFO:teuthology.orchestra.run.smithi017.stdout:Creating mgr... 2023-07-14T11:52:13.984 INFO:teuthology.orchestra.run.smithi017.stdout:Verifying port 9283 ... 2023-07-14T11:52:13.984 INFO:teuthology.orchestra.run.smithi017.stdout:Verifying port 8765 ... 2023-07-14T11:52:14.361 INFO:teuthology.orchestra.run.smithi017.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mgr.x 2023-07-14T11:52:14.361 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mgr.x.service: Unit ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mgr.x.service not loaded. 2023-07-14T11:52:14.568 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b3c386cc-223c-11ee-9b33-001a4aab830c.target.wants/ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mgr.x.service → /etc/systemd/system/ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@.service. 2023-07-14T11:52:14.647 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:14 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/4184650799' entity='client.admin' 2023-07-14T11:52:14.648 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:14 smithi017 systemd[1]: Starting Ceph mgr.x for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T11:52:15.351 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:15 smithi017 podman[126579]: 2023-07-14T11:52:15.352 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:15 smithi017 bash[126579]: 37fdbd61b165e37408b514f57bd931b2e39cae7f6e841a724b2b2e61da208f52 2023-07-14T11:52:15.380 INFO:teuthology.orchestra.run.smithi017.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-14T11:52:15.380 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stdout disabled 2023-07-14T11:52:15.397 INFO:teuthology.orchestra.run.smithi017.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-14T11:52:15.397 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stdout inactive 2023-07-14T11:52:15.397 INFO:teuthology.orchestra.run.smithi017.stdout:firewalld.service is not enabled 2023-07-14T11:52:15.397 INFO:teuthology.orchestra.run.smithi017.stdout:Not possible to enable service . firewalld.service is not available 2023-07-14T11:52:15.417 INFO:teuthology.orchestra.run.smithi017.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-14T11:52:15.418 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stdout disabled 2023-07-14T11:52:15.436 INFO:teuthology.orchestra.run.smithi017.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-14T11:52:15.437 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stdout inactive 2023-07-14T11:52:15.437 INFO:teuthology.orchestra.run.smithi017.stdout:firewalld.service is not enabled 2023-07-14T11:52:15.437 INFO:teuthology.orchestra.run.smithi017.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-07-14T11:52:15.437 INFO:teuthology.orchestra.run.smithi017.stdout:Waiting for mgr to start... 2023-07-14T11:52:15.438 INFO:teuthology.orchestra.run.smithi017.stdout:Waiting for mgr... 2023-07-14T11:52:15.645 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:15 smithi017 systemd[1]: Started Ceph mgr.x for b3c386cc-223c-11ee-9b33-001a4aab830c. 2023-07-14T11:52:15.645 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:15 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:15.550+0000 7f6022695200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-14T11:52:15.645 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:15 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:15.660+0000 7f6022695200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-14T11:52:15.897 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:15 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:15.785+0000 7f6022695200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-14T11:52:16.152 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:15 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:15.917+0000 7f6022695200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-14T11:52:16.153 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:16 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:16.149+0000 7f6022695200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-14T11:52:16.566 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:16 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/1569914022' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-14T11:52:16.639 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 2023-07-14T11:52:16.639 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout { 2023-07-14T11:52:16.639 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "fsid": "b3c386cc-223c-11ee-9b33-001a4aab830c", 2023-07-14T11:52:16.639 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "health": { 2023-07-14T11:52:16.639 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-14T11:52:16.639 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-14T11:52:16.640 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-14T11:52:16.640 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2023-07-14T11:52:16.640 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-14T11:52:16.640 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-14T11:52:16.640 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 0 2023-07-14T11:52:16.640 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout ], 2023-07-14T11:52:16.640 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-14T11:52:16.641 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "a" 2023-07-14T11:52:16.641 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout ], 2023-07-14T11:52:16.653 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2023-07-14T11:52:16.653 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-14T11:52:16.653 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T11:52:16.653 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-14T11:52:16.653 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-14T11:52:16.654 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2023-07-14T11:52:16.654 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-14T11:52:16.654 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T11:52:16.654 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-14T11:52:16.654 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-14T11:52:16.654 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-14T11:52:16.654 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-14T11:52:16.655 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-14T11:52:16.655 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-14T11:52:16.655 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2023-07-14T11:52:16.655 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-14T11:52:16.655 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-14T11:52:16.655 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-14T11:52:16.655 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-14T11:52:16.655 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-14T11:52:16.656 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-14T11:52:16.656 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-14T11:52:16.656 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-14T11:52:16.656 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-14T11:52:16.656 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2023-07-14T11:52:16.656 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-14T11:52:16.656 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T11:52:16.657 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-14T11:52:16.657 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-14T11:52:16.657 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2023-07-14T11:52:16.657 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-14T11:52:16.657 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "available": false, 2023-07-14T11:52:16.657 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-14T11:52:16.657 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-14T11:52:16.657 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-14T11:52:16.658 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-14T11:52:16.658 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "restful" 2023-07-14T11:52:16.658 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout ], 2023-07-14T11:52:16.658 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-14T11:52:16.658 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2023-07-14T11:52:16.658 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-14T11:52:16.658 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T11:52:16.659 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "modified": "2023-07-14T11:51:59.993614+0000", 2023-07-14T11:52:16.659 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-14T11:52:16.659 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2023-07-14T11:52:16.659 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-14T11:52:16.660 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout } 2023-07-14T11:52:16.660 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:16.057+0000 7fe997c03700 1 Processor -- start 2023-07-14T11:52:16.660 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:16.058+0000 7fe997c03700 1 -- start start 2023-07-14T11:52:16.660 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:16.058+0000 7fe997c03700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe99007be00 0x7fe99007a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:16.661 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:16.058+0000 7fe997c03700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99007a840 con 0x7fe99007be00 2023-07-14T11:52:16.661 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:16.059+0000 7fe99599f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe99007be00 0x7fe99007a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:16.661 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:16.059+0000 7fe99599f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe99007be00 0x7fe99007a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41160/0 (socket says 172.21.15.17:41160) 2023-07-14T11:52:16.661 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:16.059+0000 7fe99599f700 1 -- 172.21.15.17:0/3912584818 learned_addr learned my addr 172.21.15.17:0/3912584818 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:52:16.661 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:16.059+0000 7fe99599f700 1 -- 172.21.15.17:0/3912584818 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe99007a980 con 0x7fe99007be00 2023-07-14T11:52:16.661 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:16.060+0000 7fe99599f700 1 --2- 172.21.15.17:0/3912584818 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe99007be00 0x7fe99007a300 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0098a0 tx=0x7fe98c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f04f963530e252f4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:16.662 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:16.060+0000 7fe99499d700 1 -- 172.21.15.17:0/3912584818 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe98c004160 con 0x7fe99007be00 2023-07-14T11:52:16.662 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:16.060+0000 7fe99499d700 1 -- 172.21.15.17:0/3912584818 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe98c0042c0 con 0x7fe99007be00 2023-07-14T11:52:16.662 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:16.060+0000 7fe99499d700 1 -- 172.21.15.17:0/3912584818 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe98c011a00 con 0x7fe99007be00 2023-07-14T11:52:16.662 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:16.060+0000 7fe997c03700 1 -- 172.21.15.17:0/3912584818 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe99007be00 msgr2=0x7fe99007a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:16.662 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:16.060+0000 7fe997c03700 1 --2- 172.21.15.17:0/3912584818 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe99007be00 0x7fe99007a300 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0098a0 tx=0x7fe98c009bb0 comp rx=0 tx=0).stop 2023-07-14T11:52:16.662 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:16.061+0000 7fe997c03700 1 -- 172.21.15.17:0/3912584818 shutdown_connections 2023-07-14T11:52:16.662 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:16.061+0000 7fe997c03700 1 --2- 172.21.15.17:0/3912584818 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe99007be00 0x7fe99007a300 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:16.663 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:16.061+0000 7fe997c03700 1 -- 172.21.15.17:0/3912584818 >> 172.21.15.17:0/3912584818 conn(0x7fe990100970 msgr2=0x7fe990102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:16.663 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:16.061+0000 7fe997c03700 1 -- 172.21.15.17:0/3912584818 shutdown_connections 2023-07-14T11:52:16.663 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:16.061+0000 7fe997c03700 1 -- 172.21.15.17:0/3912584818 wait complete. 2023-07-14T11:52:16.663 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:16.062+0000 7fe997c03700 1 Processor -- start 2023-07-14T11:52:16.663 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:16.062+0000 7fe997c03700 1 -- start start 2023-07-14T11:52:16.663 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:16.062+0000 7fe997c03700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe99007be00 0x7fe99019b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:16.663 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:16.062+0000 7fe997c03700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99019b880 con 0x7fe99007be00 2023-07-14T11:52:16.663 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:16.063+0000 7fe99599f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe99007be00 0x7fe99019b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:16.664 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:16.063+0000 7fe99599f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe99007be00 0x7fe99019b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41170/0 (socket says 172.21.15.17:41170) 2023-07-14T11:52:16.664 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:16.063+0000 7fe99599f700 1 -- 172.21.15.17:0/1569914022 learned_addr learned my addr 172.21.15.17:0/1569914022 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:52:16.664 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:16.063+0000 7fe99599f700 1 -- 172.21.15.17:0/1569914022 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe98c009550 con 0x7fe99007be00 2023-07-14T11:52:16.664 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:16.063+0000 7fe99599f700 1 --2- 172.21.15.17:0/1569914022 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe99007be00 0x7fe99019b340 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe98c009130 tx=0x7fe98c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:16.664 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:16.064+0000 7fe986ffd700 1 -- 172.21.15.17:0/1569914022 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe98c003ab0 con 0x7fe99007be00 2023-07-14T11:52:16.664 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:16.064+0000 7fe986ffd700 1 -- 172.21.15.17:0/1569914022 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe98c003c10 con 0x7fe99007be00 2023-07-14T11:52:16.664 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:16.064+0000 7fe986ffd700 1 -- 172.21.15.17:0/1569914022 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe98c0247d0 con 0x7fe99007be00 2023-07-14T11:52:16.665 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:16.064+0000 7fe997c03700 1 -- 172.21.15.17:0/1569914022 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe99019ba80 con 0x7fe99007be00 2023-07-14T11:52:16.665 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:16.064+0000 7fe997c03700 1 -- 172.21.15.17:0/1569914022 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe99019bea0 con 0x7fe99007be00 2023-07-14T11:52:16.665 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:16.064+0000 7fe986ffd700 1 -- 172.21.15.17:0/1569914022 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe98c024d00 con 0x7fe99007be00 2023-07-14T11:52:16.665 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:16.064+0000 7fe986ffd700 1 -- 172.21.15.17:0/1569914022 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe98c0193c0 con 0x7fe99007be00 2023-07-14T11:52:16.665 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:16.065+0000 7fe997c03700 1 -- 172.21.15.17:0/1569914022 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9901952f0 con 0x7fe99007be00 2023-07-14T11:52:16.665 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:16.068+0000 7fe986ffd700 1 -- 172.21.15.17:0/1569914022 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7fe98c01abb0 con 0x7fe99007be00 2023-07-14T11:52:16.665 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:16.145+0000 7fe997c03700 1 -- 172.21.15.17:0/1569914022 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fe99002ca50 con 0x7fe99007be00 2023-07-14T11:52:16.665 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:16.146+0000 7fe986ffd700 1 -- 172.21.15.17:0/1569914022 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fe98c005a20 con 0x7fe99007be00 2023-07-14T11:52:16.666 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:16.147+0000 7fe997c03700 1 -- 172.21.15.17:0/1569914022 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe99007be00 msgr2=0x7fe99019b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:16.666 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:16.147+0000 7fe997c03700 1 --2- 172.21.15.17:0/1569914022 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe99007be00 0x7fe99019b340 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe98c009130 tx=0x7fe98c011ad0 comp rx=0 tx=0).stop 2023-07-14T11:52:16.666 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:16.147+0000 7fe997c03700 1 -- 172.21.15.17:0/1569914022 shutdown_connections 2023-07-14T11:52:16.666 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:16.147+0000 7fe997c03700 1 --2- 172.21.15.17:0/1569914022 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe99007be00 0x7fe99019b340 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:16.666 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:16.147+0000 7fe997c03700 1 -- 172.21.15.17:0/1569914022 >> 172.21.15.17:0/1569914022 conn(0x7fe990100970 msgr2=0x7fe9901020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:16.666 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:16.147+0000 7fe997c03700 1 -- 172.21.15.17:0/1569914022 shutdown_connections 2023-07-14T11:52:16.666 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:16.147+0000 7fe997c03700 1 -- 172.21.15.17:0/1569914022 wait complete. 2023-07-14T11:52:16.666 INFO:teuthology.orchestra.run.smithi017.stdout:mgr not available, waiting (1/15)... 2023-07-14T11:52:16.852 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:16 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:16.581+0000 7f6022695200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-14T11:52:16.852 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:16 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:16.702+0000 7f6022695200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-14T11:52:17.147 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:16 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:16.867+0000 7f6022695200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-14T11:52:17.148 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:16 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:16.994+0000 7f6022695200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-14T11:52:17.897 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:17 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:17.553+0000 7f6022695200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-14T11:52:18.647 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:18 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:18.357+0000 7f6022695200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-14T11:52:18.648 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:18 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:18.567+0000 7f6022695200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-14T11:52:19.841 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:19 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/1793804946' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-14T11:52:19.841 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:19 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:19.600+0000 7f6022695200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-14T11:52:19.842 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:19 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:19.729+0000 7f6022695200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-14T11:52:20.131 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 2023-07-14T11:52:20.132 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout { 2023-07-14T11:52:20.132 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "fsid": "b3c386cc-223c-11ee-9b33-001a4aab830c", 2023-07-14T11:52:20.132 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "health": { 2023-07-14T11:52:20.132 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-14T11:52:20.132 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-14T11:52:20.132 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-14T11:52:20.132 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2023-07-14T11:52:20.133 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-14T11:52:20.133 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-14T11:52:20.133 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 0 2023-07-14T11:52:20.133 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout ], 2023-07-14T11:52:20.133 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-14T11:52:20.134 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "a" 2023-07-14T11:52:20.134 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout ], 2023-07-14T11:52:20.134 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2023-07-14T11:52:20.134 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-14T11:52:20.134 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T11:52:20.134 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-14T11:52:20.134 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-14T11:52:20.134 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2023-07-14T11:52:20.135 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-14T11:52:20.135 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T11:52:20.135 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-14T11:52:20.135 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-14T11:52:20.135 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-14T11:52:20.141 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-14T11:52:20.141 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-14T11:52:20.141 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-14T11:52:20.141 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2023-07-14T11:52:20.141 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-14T11:52:20.141 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-14T11:52:20.142 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-14T11:52:20.142 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-14T11:52:20.142 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-14T11:52:20.142 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-14T11:52:20.142 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-14T11:52:20.142 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-14T11:52:20.142 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-14T11:52:20.142 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2023-07-14T11:52:20.143 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-14T11:52:20.143 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T11:52:20.143 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-14T11:52:20.143 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-14T11:52:20.143 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2023-07-14T11:52:20.143 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-14T11:52:20.143 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "available": false, 2023-07-14T11:52:20.144 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-14T11:52:20.144 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-14T11:52:20.144 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-14T11:52:20.144 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-14T11:52:20.144 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "restful" 2023-07-14T11:52:20.144 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout ], 2023-07-14T11:52:20.144 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-14T11:52:20.144 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2023-07-14T11:52:20.145 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-14T11:52:20.145 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T11:52:20.145 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "modified": "2023-07-14T11:51:59.993614+0000", 2023-07-14T11:52:20.145 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-14T11:52:20.145 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2023-07-14T11:52:20.145 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-14T11:52:20.145 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout } 2023-07-14T11:52:20.146 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:19.578+0000 7fbf8e39a700 1 Processor -- start 2023-07-14T11:52:20.146 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:19.578+0000 7fbf8e39a700 1 -- start start 2023-07-14T11:52:20.146 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:19.578+0000 7fbf8e39a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf8807be00 0x7fbf8807a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:20.146 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:19.578+0000 7fbf8e39a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8807a840 con 0x7fbf8807be00 2023-07-14T11:52:20.146 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:19.578+0000 7fbf87fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf8807be00 0x7fbf8807a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:20.146 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:19.578+0000 7fbf87fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf8807be00 0x7fbf8807a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41184/0 (socket says 172.21.15.17:41184) 2023-07-14T11:52:20.147 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:19.578+0000 7fbf87fff700 1 -- 172.21.15.17:0/486652475 learned_addr learned my addr 172.21.15.17:0/486652475 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:52:20.147 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:19.578+0000 7fbf87fff700 1 -- 172.21.15.17:0/486652475 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf8807a980 con 0x7fbf8807be00 2023-07-14T11:52:20.147 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:19.578+0000 7fbf87fff700 1 --2- 172.21.15.17:0/486652475 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf8807be00 0x7fbf8807a300 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbf780098a0 tx=0x7fbf78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e3e3e55f7e6644f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:20.147 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:19.579+0000 7fbf86ffd700 1 -- 172.21.15.17:0/486652475 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf78004160 con 0x7fbf8807be00 2023-07-14T11:52:20.147 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:19.579+0000 7fbf86ffd700 1 -- 172.21.15.17:0/486652475 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbf780042c0 con 0x7fbf8807be00 2023-07-14T11:52:20.147 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:19.579+0000 7fbf86ffd700 1 -- 172.21.15.17:0/486652475 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf78011a00 con 0x7fbf8807be00 2023-07-14T11:52:20.147 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:19.579+0000 7fbf8e39a700 1 -- 172.21.15.17:0/486652475 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf8807be00 msgr2=0x7fbf8807a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:20.147 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:19.579+0000 7fbf8e39a700 1 --2- 172.21.15.17:0/486652475 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf8807be00 0x7fbf8807a300 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbf780098a0 tx=0x7fbf78009bb0 comp rx=0 tx=0).stop 2023-07-14T11:52:20.148 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:19.579+0000 7fbf8e39a700 1 -- 172.21.15.17:0/486652475 shutdown_connections 2023-07-14T11:52:20.148 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:19.579+0000 7fbf8e39a700 1 --2- 172.21.15.17:0/486652475 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf8807be00 0x7fbf8807a300 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:20.148 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:19.579+0000 7fbf8e39a700 1 -- 172.21.15.17:0/486652475 >> 172.21.15.17:0/486652475 conn(0x7fbf88100970 msgr2=0x7fbf88102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:20.148 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:19.580+0000 7fbf8e39a700 1 -- 172.21.15.17:0/486652475 shutdown_connections 2023-07-14T11:52:20.148 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:19.580+0000 7fbf8e39a700 1 -- 172.21.15.17:0/486652475 wait complete. 2023-07-14T11:52:20.148 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:19.580+0000 7fbf8e39a700 1 Processor -- start 2023-07-14T11:52:20.148 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:19.581+0000 7fbf8e39a700 1 -- start start 2023-07-14T11:52:20.149 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:19.581+0000 7fbf8e39a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf8807be00 0x7fbf8819b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:20.149 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:19.581+0000 7fbf8e39a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8819b880 con 0x7fbf8807be00 2023-07-14T11:52:20.149 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:19.581+0000 7fbf87fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf8807be00 0x7fbf8819b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:20.149 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:19.581+0000 7fbf87fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf8807be00 0x7fbf8819b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41192/0 (socket says 172.21.15.17:41192) 2023-07-14T11:52:20.149 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:19.581+0000 7fbf87fff700 1 -- 172.21.15.17:0/1793804946 learned_addr learned my addr 172.21.15.17:0/1793804946 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:52:20.149 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:19.582+0000 7fbf87fff700 1 -- 172.21.15.17:0/1793804946 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf78009550 con 0x7fbf8807be00 2023-07-14T11:52:20.149 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:19.582+0000 7fbf87fff700 1 --2- 172.21.15.17:0/1793804946 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf8807be00 0x7fbf8819b340 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbf78009130 tx=0x7fbf78011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:20.149 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:19.582+0000 7fbf857fa700 1 -- 172.21.15.17:0/1793804946 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf78003ab0 con 0x7fbf8807be00 2023-07-14T11:52:20.150 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:19.582+0000 7fbf857fa700 1 -- 172.21.15.17:0/1793804946 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbf78003c10 con 0x7fbf8807be00 2023-07-14T11:52:20.150 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:19.582+0000 7fbf8e39a700 1 -- 172.21.15.17:0/1793804946 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf8819ba80 con 0x7fbf8807be00 2023-07-14T11:52:20.150 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:19.582+0000 7fbf857fa700 1 -- 172.21.15.17:0/1793804946 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf780247d0 con 0x7fbf8807be00 2023-07-14T11:52:20.150 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:19.583+0000 7fbf8e39a700 1 -- 172.21.15.17:0/1793804946 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf8819bea0 con 0x7fbf8807be00 2023-07-14T11:52:20.150 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:19.583+0000 7fbf857fa700 1 -- 172.21.15.17:0/1793804946 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fbf78024d00 con 0x7fbf8807be00 2023-07-14T11:52:20.150 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:19.583+0000 7fbf857fa700 1 -- 172.21.15.17:0/1793804946 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fbf780193c0 con 0x7fbf8807be00 2023-07-14T11:52:20.150 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:19.584+0000 7fbf8e39a700 1 -- 172.21.15.17:0/1793804946 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf881952f0 con 0x7fbf8807be00 2023-07-14T11:52:20.151 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:19.587+0000 7fbf857fa700 1 -- 172.21.15.17:0/1793804946 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7fbf7801abb0 con 0x7fbf8807be00 2023-07-14T11:52:20.151 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:19.664+0000 7fbf8e39a700 1 -- 172.21.15.17:0/1793804946 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fbf8802ca50 con 0x7fbf8807be00 2023-07-14T11:52:20.151 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:19.664+0000 7fbf857fa700 1 -- 172.21.15.17:0/1793804946 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fbf78005a20 con 0x7fbf8807be00 2023-07-14T11:52:20.151 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:19.665+0000 7fbf8e39a700 1 -- 172.21.15.17:0/1793804946 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf8807be00 msgr2=0x7fbf8819b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:20.151 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:19.665+0000 7fbf8e39a700 1 --2- 172.21.15.17:0/1793804946 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf8807be00 0x7fbf8819b340 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbf78009130 tx=0x7fbf78011ad0 comp rx=0 tx=0).stop 2023-07-14T11:52:20.151 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:19.666+0000 7fbf8e39a700 1 -- 172.21.15.17:0/1793804946 shutdown_connections 2023-07-14T11:52:20.151 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:19.666+0000 7fbf8e39a700 1 --2- 172.21.15.17:0/1793804946 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf8807be00 0x7fbf8819b340 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:20.151 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:19.666+0000 7fbf8e39a700 1 -- 172.21.15.17:0/1793804946 >> 172.21.15.17:0/1793804946 conn(0x7fbf88100970 msgr2=0x7fbf881020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:20.152 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:19.666+0000 7fbf8e39a700 1 -- 172.21.15.17:0/1793804946 shutdown_connections 2023-07-14T11:52:20.152 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:19.666+0000 7fbf8e39a700 1 -- 172.21.15.17:0/1793804946 wait complete. 2023-07-14T11:52:20.152 INFO:teuthology.orchestra.run.smithi017.stdout:mgr not available, waiting (2/15)... 2023-07-14T11:52:20.152 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:19 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:19.962+0000 7f6022695200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-14T11:52:20.595 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:20 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:20.332+0000 7f6022695200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-14T11:52:20.897 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:20 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:20.611+0000 7f6022695200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-14T11:52:20.898 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:20 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:20.810+0000 7f6022695200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-14T11:52:21.397 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:21 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:21.162+0000 7f6022695200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-14T11:52:22.397 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:22 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:22.113+0000 7f6022695200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-14T11:52:22.397 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:22 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:22.296+0000 7f6022695200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-14T11:52:22.829 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:22 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:22.555+0000 7f6022695200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-14T11:52:22.829 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:22 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:22.713+0000 7f6022695200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-14T11:52:23.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:22 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/3945843121' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-14T11:52:23.148 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:23 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:23.012+0000 7f6022695200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-14T11:52:23.417 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 2023-07-14T11:52:23.418 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout { 2023-07-14T11:52:23.418 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "fsid": "b3c386cc-223c-11ee-9b33-001a4aab830c", 2023-07-14T11:52:23.418 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "health": { 2023-07-14T11:52:23.418 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-14T11:52:23.418 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-14T11:52:23.419 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-14T11:52:23.419 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2023-07-14T11:52:23.419 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-14T11:52:23.419 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-14T11:52:23.419 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 0 2023-07-14T11:52:23.419 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout ], 2023-07-14T11:52:23.419 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-14T11:52:23.420 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "a" 2023-07-14T11:52:23.420 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout ], 2023-07-14T11:52:23.420 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2023-07-14T11:52:23.420 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-14T11:52:23.420 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T11:52:23.420 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-14T11:52:23.420 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-14T11:52:23.421 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2023-07-14T11:52:23.421 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-14T11:52:23.421 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T11:52:23.421 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-14T11:52:23.421 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-14T11:52:23.427 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-14T11:52:23.427 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-14T11:52:23.427 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-14T11:52:23.427 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-14T11:52:23.427 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2023-07-14T11:52:23.427 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-14T11:52:23.428 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-14T11:52:23.428 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-14T11:52:23.428 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-14T11:52:23.428 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-14T11:52:23.428 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-14T11:52:23.428 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-14T11:52:23.428 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-14T11:52:23.429 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-14T11:52:23.429 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2023-07-14T11:52:23.429 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-14T11:52:23.429 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T11:52:23.429 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-14T11:52:23.429 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-14T11:52:23.429 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2023-07-14T11:52:23.430 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-14T11:52:23.430 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "available": false, 2023-07-14T11:52:23.430 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-14T11:52:23.430 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-14T11:52:23.431 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-14T11:52:23.431 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-14T11:52:23.431 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "restful" 2023-07-14T11:52:23.431 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout ], 2023-07-14T11:52:23.431 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-14T11:52:23.431 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2023-07-14T11:52:23.431 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-14T11:52:23.432 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T11:52:23.432 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "modified": "2023-07-14T11:51:59.993614+0000", 2023-07-14T11:52:23.432 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-14T11:52:23.432 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2023-07-14T11:52:23.432 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-14T11:52:23.432 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout } 2023-07-14T11:52:23.432 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:22.737+0000 7f9e5f522700 1 Processor -- start 2023-07-14T11:52:23.432 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:22.737+0000 7f9e5f522700 1 -- start start 2023-07-14T11:52:23.433 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:22.738+0000 7f9e5f522700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e5807be20 0x7f9e5807a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:23.433 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:22.738+0000 7f9e5f522700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5807a860 con 0x7f9e5807be20 2023-07-14T11:52:23.433 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:22.738+0000 7f9e5d2be700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e5807be20 0x7f9e5807a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:23.433 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:22.738+0000 7f9e5d2be700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e5807be20 0x7f9e5807a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41206/0 (socket says 172.21.15.17:41206) 2023-07-14T11:52:23.433 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:22.738+0000 7f9e5d2be700 1 -- 172.21.15.17:0/2339054507 learned_addr learned my addr 172.21.15.17:0/2339054507 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:52:23.433 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:22.738+0000 7f9e5d2be700 1 -- 172.21.15.17:0/2339054507 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e5807a9a0 con 0x7f9e5807be20 2023-07-14T11:52:23.433 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:22.739+0000 7f9e5d2be700 1 --2- 172.21.15.17:0/2339054507 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e5807be20 0x7f9e5807a320 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9e540098a0 tx=0x7f9e54009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2f6448d8047a249 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:23.434 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:22.739+0000 7f9e4ffff700 1 -- 172.21.15.17:0/2339054507 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e54004160 con 0x7f9e5807be20 2023-07-14T11:52:23.434 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:22.739+0000 7f9e4ffff700 1 -- 172.21.15.17:0/2339054507 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9e540042c0 con 0x7f9e5807be20 2023-07-14T11:52:23.434 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:22.739+0000 7f9e4ffff700 1 -- 172.21.15.17:0/2339054507 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e54011a00 con 0x7f9e5807be20 2023-07-14T11:52:23.434 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:22.740+0000 7f9e5f522700 1 -- 172.21.15.17:0/2339054507 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e5807be20 msgr2=0x7f9e5807a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:23.434 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:22.740+0000 7f9e5f522700 1 --2- 172.21.15.17:0/2339054507 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e5807be20 0x7f9e5807a320 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9e540098a0 tx=0x7f9e54009bb0 comp rx=0 tx=0).stop 2023-07-14T11:52:23.434 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:22.740+0000 7f9e5f522700 1 -- 172.21.15.17:0/2339054507 shutdown_connections 2023-07-14T11:52:23.434 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:22.740+0000 7f9e5f522700 1 --2- 172.21.15.17:0/2339054507 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e5807be20 0x7f9e5807a320 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:23.434 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:22.740+0000 7f9e5f522700 1 -- 172.21.15.17:0/2339054507 >> 172.21.15.17:0/2339054507 conn(0x7f9e58100970 msgr2=0x7f9e58102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:23.435 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:22.740+0000 7f9e5f522700 1 -- 172.21.15.17:0/2339054507 shutdown_connections 2023-07-14T11:52:23.435 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:22.740+0000 7f9e5f522700 1 -- 172.21.15.17:0/2339054507 wait complete. 2023-07-14T11:52:23.435 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:22.741+0000 7f9e5f522700 1 Processor -- start 2023-07-14T11:52:23.435 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:22.741+0000 7f9e5f522700 1 -- start start 2023-07-14T11:52:23.435 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:22.741+0000 7f9e5f522700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e5807be20 0x7f9e5819b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:23.435 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:22.741+0000 7f9e5f522700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5819b880 con 0x7f9e5807be20 2023-07-14T11:52:23.435 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:22.742+0000 7f9e5d2be700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e5807be20 0x7f9e5819b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:23.436 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:22.742+0000 7f9e5d2be700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e5807be20 0x7f9e5819b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41216/0 (socket says 172.21.15.17:41216) 2023-07-14T11:52:23.436 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:22.742+0000 7f9e5d2be700 1 -- 172.21.15.17:0/3945843121 learned_addr learned my addr 172.21.15.17:0/3945843121 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:52:23.436 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:22.742+0000 7f9e5d2be700 1 -- 172.21.15.17:0/3945843121 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e54009550 con 0x7f9e5807be20 2023-07-14T11:52:23.436 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:22.742+0000 7f9e5d2be700 1 --2- 172.21.15.17:0/3945843121 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e5807be20 0x7f9e5819b340 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9e54009fb0 tx=0x7f9e54011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:23.436 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:22.742+0000 7f9e4e7fc700 1 -- 172.21.15.17:0/3945843121 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e540039c0 con 0x7f9e5807be20 2023-07-14T11:52:23.436 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:22.742+0000 7f9e5f522700 1 -- 172.21.15.17:0/3945843121 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e5819ba80 con 0x7f9e5807be20 2023-07-14T11:52:23.436 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:22.742+0000 7f9e4e7fc700 1 -- 172.21.15.17:0/3945843121 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9e54003b20 con 0x7f9e5807be20 2023-07-14T11:52:23.437 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:22.742+0000 7f9e4e7fc700 1 -- 172.21.15.17:0/3945843121 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e5401a5e0 con 0x7f9e5807be20 2023-07-14T11:52:23.437 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:22.743+0000 7f9e5f522700 1 -- 172.21.15.17:0/3945843121 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e5819bea0 con 0x7f9e5807be20 2023-07-14T11:52:23.437 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:22.743+0000 7f9e4e7fc700 1 -- 172.21.15.17:0/3945843121 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9e5401ab50 con 0x7f9e5807be20 2023-07-14T11:52:23.437 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:22.743+0000 7f9e4e7fc700 1 -- 172.21.15.17:0/3945843121 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9e5402ddd0 con 0x7f9e5807be20 2023-07-14T11:52:23.437 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:22.744+0000 7f9e5f522700 1 -- 172.21.15.17:0/3945843121 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e58195320 con 0x7f9e5807be20 2023-07-14T11:52:23.437 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:22.746+0000 7f9e4e7fc700 1 -- 172.21.15.17:0/3945843121 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f9e54003c90 con 0x7f9e5807be20 2023-07-14T11:52:23.437 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:22.824+0000 7f9e5f522700 1 -- 172.21.15.17:0/3945843121 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f9e5802c850 con 0x7f9e5807be20 2023-07-14T11:52:23.437 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:22.824+0000 7f9e4e7fc700 1 -- 172.21.15.17:0/3945843121 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f9e54024430 con 0x7f9e5807be20 2023-07-14T11:52:23.438 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:22.825+0000 7f9e5f522700 1 -- 172.21.15.17:0/3945843121 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e5807be20 msgr2=0x7f9e5819b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:23.438 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:22.825+0000 7f9e5f522700 1 --2- 172.21.15.17:0/3945843121 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e5807be20 0x7f9e5819b340 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9e54009fb0 tx=0x7f9e54011ad0 comp rx=0 tx=0).stop 2023-07-14T11:52:23.438 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:22.825+0000 7f9e5f522700 1 -- 172.21.15.17:0/3945843121 shutdown_connections 2023-07-14T11:52:23.438 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:22.825+0000 7f9e5f522700 1 --2- 172.21.15.17:0/3945843121 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e5807be20 0x7f9e5819b340 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:23.438 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:22.825+0000 7f9e5f522700 1 -- 172.21.15.17:0/3945843121 >> 172.21.15.17:0/3945843121 conn(0x7f9e58100970 msgr2=0x7f9e58102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:23.438 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:22.826+0000 7f9e5f522700 1 -- 172.21.15.17:0/3945843121 shutdown_connections 2023-07-14T11:52:23.438 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:22.826+0000 7f9e5f522700 1 -- 172.21.15.17:0/3945843121 wait complete. 2023-07-14T11:52:23.439 INFO:teuthology.orchestra.run.smithi017.stdout:mgr not available, waiting (3/15)... 2023-07-14T11:52:24.397 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:24 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:24.127+0000 7f6022695200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-14T11:52:24.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:24 smithi017 ceph-mon[126339]: Activating manager daemon x 2023-07-14T11:52:24.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:24 smithi017 ceph-mon[126339]: mgrmap e2: x(active, starting, since 0.00457367s) 2023-07-14T11:52:24.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:24 smithi017 ceph-mon[126339]: from='mgr.14100 172.21.15.17:0/3982650973' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-14T11:52:24.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:24 smithi017 ceph-mon[126339]: from='mgr.14100 172.21.15.17:0/3982650973' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-14T11:52:24.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:24 smithi017 ceph-mon[126339]: from='mgr.14100 172.21.15.17:0/3982650973' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-14T11:52:24.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:24 smithi017 ceph-mon[126339]: from='mgr.14100 172.21.15.17:0/3982650973' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T11:52:24.399 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:24 smithi017 ceph-mon[126339]: from='mgr.14100 172.21.15.17:0/3982650973' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-14T11:52:24.399 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:24 smithi017 ceph-mon[126339]: Manager daemon x is now available 2023-07-14T11:52:24.399 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:24 smithi017 ceph-mon[126339]: from='mgr.14100 172.21.15.17:0/3982650973' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-14T11:52:24.399 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:24 smithi017 ceph-mon[126339]: from='mgr.14100 172.21.15.17:0/3982650973' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-14T11:52:24.399 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:24 smithi017 ceph-mon[126339]: from='mgr.14100 172.21.15.17:0/3982650973' entity='mgr.x' 2023-07-14T11:52:24.399 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:24 smithi017 ceph-mon[126339]: from='mgr.14100 172.21.15.17:0/3982650973' entity='mgr.x' 2023-07-14T11:52:24.400 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:24 smithi017 ceph-mon[126339]: from='mgr.14100 172.21.15.17:0/3982650973' entity='mgr.x' 2023-07-14T11:52:26.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:26 smithi017 ceph-mon[126339]: mgrmap e3: x(active, since 1.00999s) 2023-07-14T11:52:26.813 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 2023-07-14T11:52:26.815 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout { 2023-07-14T11:52:26.815 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "fsid": "b3c386cc-223c-11ee-9b33-001a4aab830c", 2023-07-14T11:52:26.816 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "health": { 2023-07-14T11:52:26.816 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-14T11:52:26.816 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-14T11:52:26.816 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-14T11:52:26.816 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2023-07-14T11:52:26.816 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-14T11:52:26.816 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-14T11:52:26.816 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 0 2023-07-14T11:52:26.817 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout ], 2023-07-14T11:52:26.817 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-14T11:52:26.817 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "a" 2023-07-14T11:52:26.817 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout ], 2023-07-14T11:52:26.817 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "quorum_age": 13, 2023-07-14T11:52:26.817 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-14T11:52:26.817 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T11:52:26.818 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-14T11:52:26.818 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-14T11:52:26.818 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2023-07-14T11:52:26.818 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-14T11:52:26.818 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T11:52:26.818 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-14T11:52:26.818 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-14T11:52:26.818 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-14T11:52:26.824 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-14T11:52:26.825 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-14T11:52:26.825 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-14T11:52:26.825 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2023-07-14T11:52:26.825 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-14T11:52:26.825 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-14T11:52:26.825 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-14T11:52:26.825 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-14T11:52:26.825 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-14T11:52:26.826 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-14T11:52:26.826 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-14T11:52:26.826 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-14T11:52:26.826 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-14T11:52:26.826 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2023-07-14T11:52:26.826 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-14T11:52:26.826 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T11:52:26.827 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-14T11:52:26.827 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-14T11:52:26.827 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2023-07-14T11:52:26.827 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-14T11:52:26.827 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "available": true, 2023-07-14T11:52:26.827 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-14T11:52:26.827 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-14T11:52:26.827 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-14T11:52:26.828 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-14T11:52:26.828 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "restful" 2023-07-14T11:52:26.828 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout ], 2023-07-14T11:52:26.828 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-14T11:52:26.828 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2023-07-14T11:52:26.828 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-14T11:52:26.828 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T11:52:26.828 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "modified": "2023-07-14T11:51:59.993614+0000", 2023-07-14T11:52:26.829 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-14T11:52:26.829 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout }, 2023-07-14T11:52:26.829 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-14T11:52:26.829 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout } 2023-07-14T11:52:26.829 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.018+0000 7fe0441f2700 1 Processor -- start 2023-07-14T11:52:26.829 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.018+0000 7fe0441f2700 1 -- start start 2023-07-14T11:52:26.829 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.019+0000 7fe0441f2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe03c07bde0 0x7fe03c07a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:26.830 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.019+0000 7fe0441f2700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c07a820 con 0x7fe03c07bde0 2023-07-14T11:52:26.830 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.019+0000 7fe041f8e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe03c07bde0 0x7fe03c07a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:26.830 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.019+0000 7fe041f8e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe03c07bde0 0x7fe03c07a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53310/0 (socket says 172.21.15.17:53310) 2023-07-14T11:52:26.830 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.019+0000 7fe041f8e700 1 -- 172.21.15.17:0/1354567990 learned_addr learned my addr 172.21.15.17:0/1354567990 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:52:26.830 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.020+0000 7fe041f8e700 1 -- 172.21.15.17:0/1354567990 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe03c07a960 con 0x7fe03c07bde0 2023-07-14T11:52:26.830 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.020+0000 7fe041f8e700 1 --2- 172.21.15.17:0/1354567990 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe03c07bde0 0x7fe03c07a2e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe0380098a0 tx=0x7fe038009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9931e7b4d2010201 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:26.830 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.020+0000 7fe040f8c700 1 -- 172.21.15.17:0/1354567990 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe038004160 con 0x7fe03c07bde0 2023-07-14T11:52:26.830 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.020+0000 7fe040f8c700 1 -- 172.21.15.17:0/1354567990 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe0380042c0 con 0x7fe03c07bde0 2023-07-14T11:52:26.831 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.020+0000 7fe040f8c700 1 -- 172.21.15.17:0/1354567990 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe038011a00 con 0x7fe03c07bde0 2023-07-14T11:52:26.831 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.021+0000 7fe0441f2700 1 -- 172.21.15.17:0/1354567990 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe03c07bde0 msgr2=0x7fe03c07a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:26.831 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.021+0000 7fe0441f2700 1 --2- 172.21.15.17:0/1354567990 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe03c07bde0 0x7fe03c07a2e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe0380098a0 tx=0x7fe038009bb0 comp rx=0 tx=0).stop 2023-07-14T11:52:26.831 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.021+0000 7fe0441f2700 1 -- 172.21.15.17:0/1354567990 shutdown_connections 2023-07-14T11:52:26.831 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.021+0000 7fe0441f2700 1 --2- 172.21.15.17:0/1354567990 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe03c07bde0 0x7fe03c07a2e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:26.831 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.021+0000 7fe0441f2700 1 -- 172.21.15.17:0/1354567990 >> 172.21.15.17:0/1354567990 conn(0x7fe03c100970 msgr2=0x7fe03c102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:26.831 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.021+0000 7fe0441f2700 1 -- 172.21.15.17:0/1354567990 shutdown_connections 2023-07-14T11:52:26.831 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.021+0000 7fe0441f2700 1 -- 172.21.15.17:0/1354567990 wait complete. 2023-07-14T11:52:26.832 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.022+0000 7fe0441f2700 1 Processor -- start 2023-07-14T11:52:26.832 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.022+0000 7fe0441f2700 1 -- start start 2023-07-14T11:52:26.832 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.022+0000 7fe0441f2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe03c07bde0 0x7fe03c19b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:26.832 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.022+0000 7fe0441f2700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c19b850 con 0x7fe03c07bde0 2023-07-14T11:52:26.832 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.023+0000 7fe041f8e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe03c07bde0 0x7fe03c19b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:26.832 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.023+0000 7fe041f8e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe03c07bde0 0x7fe03c19b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53324/0 (socket says 172.21.15.17:53324) 2023-07-14T11:52:26.832 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.023+0000 7fe041f8e700 1 -- 172.21.15.17:0/2019824421 learned_addr learned my addr 172.21.15.17:0/2019824421 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:52:26.833 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.023+0000 7fe041f8e700 1 -- 172.21.15.17:0/2019824421 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe038009550 con 0x7fe03c07bde0 2023-07-14T11:52:26.833 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.023+0000 7fe041f8e700 1 --2- 172.21.15.17:0/2019824421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe03c07bde0 0x7fe03c19b310 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe038009130 tx=0x7fe038011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:26.833 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.024+0000 7fe032ffd700 1 -- 172.21.15.17:0/2019824421 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe038003ab0 con 0x7fe03c07bde0 2023-07-14T11:52:26.833 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.024+0000 7fe032ffd700 1 -- 172.21.15.17:0/2019824421 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe038003c10 con 0x7fe03c07bde0 2023-07-14T11:52:26.833 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.024+0000 7fe0441f2700 1 -- 172.21.15.17:0/2019824421 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe03c19ba50 con 0x7fe03c07bde0 2023-07-14T11:52:26.833 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.024+0000 7fe032ffd700 1 -- 172.21.15.17:0/2019824421 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe0380247d0 con 0x7fe03c07bde0 2023-07-14T11:52:26.833 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.024+0000 7fe0441f2700 1 -- 172.21.15.17:0/2019824421 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe03c19be70 con 0x7fe03c07bde0 2023-07-14T11:52:26.833 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.025+0000 7fe032ffd700 1 -- 172.21.15.17:0/2019824421 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 3) v1 ==== 45366+0+0 (secure 0 0 0) 0x7fe03801a430 con 0x7fe03c07bde0 2023-07-14T11:52:26.834 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.025+0000 7fe0441f2700 1 -- 172.21.15.17:0/2019824421 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe03c1952f0 con 0x7fe03c07bde0 2023-07-14T11:52:26.834 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.025+0000 7fe032ffd700 1 --2- 172.21.15.17:0/2019824421 >> [v2:172.21.15.17:6800/2804227742,v1:172.21.15.17:6801/2804227742] conn(0x7fe0280385e0 0x7fe02803aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:26.834 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.025+0000 7fe032ffd700 1 -- 172.21.15.17:0/2019824421 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe03804c250 con 0x7fe03c07bde0 2023-07-14T11:52:26.834 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.025+0000 7fe04178d700 1 --2- 172.21.15.17:0/2019824421 >> [v2:172.21.15.17:6800/2804227742,v1:172.21.15.17:6801/2804227742] conn(0x7fe0280385e0 0x7fe02803aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:26.834 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.026+0000 7fe04178d700 1 --2- 172.21.15.17:0/2019824421 >> [v2:172.21.15.17:6800/2804227742,v1:172.21.15.17:6801/2804227742] conn(0x7fe0280385e0 0x7fe02803aa90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe02c006fd0 tx=0x7fe02c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:26.834 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.031+0000 7fe032ffd700 1 -- 172.21.15.17:0/2019824421 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe03801ad90 con 0x7fe03c07bde0 2023-07-14T11:52:26.834 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.139+0000 7fe032ffd700 1 -- 172.21.15.17:0/2019824421 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7fe03801a6e0 con 0x7fe03c07bde0 2023-07-14T11:52:26.834 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.327+0000 7fe0441f2700 1 -- 172.21.15.17:0/2019824421 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fe03c02c850 con 0x7fe03c07bde0 2023-07-14T11:52:26.835 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.327+0000 7fe032ffd700 1 -- 172.21.15.17:0/2019824421 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fe038018740 con 0x7fe03c07bde0 2023-07-14T11:52:26.835 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.330+0000 7fe0441f2700 1 -- 172.21.15.17:0/2019824421 >> [v2:172.21.15.17:6800/2804227742,v1:172.21.15.17:6801/2804227742] conn(0x7fe0280385e0 msgr2=0x7fe02803aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:26.835 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.330+0000 7fe0441f2700 1 --2- 172.21.15.17:0/2019824421 >> [v2:172.21.15.17:6800/2804227742,v1:172.21.15.17:6801/2804227742] conn(0x7fe0280385e0 0x7fe02803aa90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe02c006fd0 tx=0x7fe02c006e40 comp rx=0 tx=0).stop 2023-07-14T11:52:26.835 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.330+0000 7fe0441f2700 1 -- 172.21.15.17:0/2019824421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe03c07bde0 msgr2=0x7fe03c19b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:26.835 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.330+0000 7fe0441f2700 1 --2- 172.21.15.17:0/2019824421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe03c07bde0 0x7fe03c19b310 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe038009130 tx=0x7fe038011ad0 comp rx=0 tx=0).stop 2023-07-14T11:52:26.835 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.330+0000 7fe0441f2700 1 -- 172.21.15.17:0/2019824421 shutdown_connections 2023-07-14T11:52:26.835 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.330+0000 7fe0441f2700 1 --2- 172.21.15.17:0/2019824421 >> [v2:172.21.15.17:6800/2804227742,v1:172.21.15.17:6801/2804227742] conn(0x7fe0280385e0 0x7fe02803aa90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:26.835 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.330+0000 7fe0441f2700 1 --2- 172.21.15.17:0/2019824421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe03c07bde0 0x7fe03c19b310 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:26.836 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.330+0000 7fe0441f2700 1 -- 172.21.15.17:0/2019824421 >> 172.21.15.17:0/2019824421 conn(0x7fe03c100970 msgr2=0x7fe03c102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:26.836 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.330+0000 7fe0441f2700 1 -- 172.21.15.17:0/2019824421 shutdown_connections 2023-07-14T11:52:26.836 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:26.330+0000 7fe0441f2700 1 -- 172.21.15.17:0/2019824421 wait complete. 2023-07-14T11:52:26.836 INFO:teuthology.orchestra.run.smithi017.stdout:mgr is available 2023-07-14T11:52:27.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:27 smithi017 ceph-mon[126339]: mgrmap e4: x(active, since 2s) 2023-07-14T11:52:27.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:27 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/2019824421' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-14T11:52:28.096 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 2023-07-14T11:52:28.097 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout [global] 2023-07-14T11:52:28.097 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout fsid = b3c386cc-223c-11ee-9b33-001a4aab830c 2023-07-14T11:52:28.097 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-07-14T11:52:28.097 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-07-14T11:52:28.097 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-07-14T11:52:28.097 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-07-14T11:52:28.101 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 2023-07-14T11:52:28.102 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout [mgr] 2023-07-14T11:52:28.102 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-07-14T11:52:28.102 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 2023-07-14T11:52:28.102 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout [osd] 2023-07-14T11:52:28.102 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-07-14T11:52:28.102 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-07-14T11:52:28.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-07-14T11:52:28.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-07-14T11:52:28.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-07-14T11:52:28.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.420+0000 7f859d7c8700 1 Processor -- start 2023-07-14T11:52:28.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.420+0000 7f859d7c8700 1 -- start start 2023-07-14T11:52:28.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.420+0000 7f859d7c8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f859807be00 0x7f859807a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:28.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.420+0000 7f859d7c8700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f859807a840 con 0x7f859807be00 2023-07-14T11:52:28.103 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.420+0000 7f8596ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f859807be00 0x7f859807a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:28.104 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.420+0000 7f8596ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f859807be00 0x7f859807a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53334/0 (socket says 172.21.15.17:53334) 2023-07-14T11:52:28.104 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.420+0000 7f8596ffd700 1 -- 172.21.15.17:0/3238478447 learned_addr learned my addr 172.21.15.17:0/3238478447 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:52:28.104 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.421+0000 7f8596ffd700 1 -- 172.21.15.17:0/3238478447 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f859807a980 con 0x7f859807be00 2023-07-14T11:52:28.104 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.421+0000 7f8596ffd700 1 --2- 172.21.15.17:0/3238478447 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f859807be00 0x7f859807a300 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f85880098a0 tx=0x7f8588009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89b6f51aee52039c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:28.104 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.421+0000 7f8595ffb700 1 -- 172.21.15.17:0/3238478447 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8588004160 con 0x7f859807be00 2023-07-14T11:52:28.104 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.421+0000 7f8595ffb700 1 -- 172.21.15.17:0/3238478447 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f85880042c0 con 0x7f859807be00 2023-07-14T11:52:28.104 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.421+0000 7f8595ffb700 1 -- 172.21.15.17:0/3238478447 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8588011a00 con 0x7f859807be00 2023-07-14T11:52:28.105 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.422+0000 7f859d7c8700 1 -- 172.21.15.17:0/3238478447 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f859807be00 msgr2=0x7f859807a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:28.105 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.422+0000 7f859d7c8700 1 --2- 172.21.15.17:0/3238478447 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f859807be00 0x7f859807a300 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f85880098a0 tx=0x7f8588009bb0 comp rx=0 tx=0).stop 2023-07-14T11:52:28.105 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.422+0000 7f859d7c8700 1 -- 172.21.15.17:0/3238478447 shutdown_connections 2023-07-14T11:52:28.105 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.422+0000 7f859d7c8700 1 --2- 172.21.15.17:0/3238478447 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f859807be00 0x7f859807a300 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:28.105 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.422+0000 7f859d7c8700 1 -- 172.21.15.17:0/3238478447 >> 172.21.15.17:0/3238478447 conn(0x7f8598100970 msgr2=0x7f8598102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:28.105 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.422+0000 7f859d7c8700 1 -- 172.21.15.17:0/3238478447 shutdown_connections 2023-07-14T11:52:28.105 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.422+0000 7f859d7c8700 1 -- 172.21.15.17:0/3238478447 wait complete. 2023-07-14T11:52:28.105 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.423+0000 7f859d7c8700 1 Processor -- start 2023-07-14T11:52:28.106 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.423+0000 7f859d7c8700 1 -- start start 2023-07-14T11:52:28.106 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.424+0000 7f859d7c8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f859807be00 0x7f859819b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:28.106 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.424+0000 7f859d7c8700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f859819b880 con 0x7f859807be00 2023-07-14T11:52:28.106 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.424+0000 7f8596ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f859807be00 0x7f859819b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:28.106 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.424+0000 7f8596ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f859807be00 0x7f859819b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53346/0 (socket says 172.21.15.17:53346) 2023-07-14T11:52:28.106 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.424+0000 7f8596ffd700 1 -- 172.21.15.17:0/2791389381 learned_addr learned my addr 172.21.15.17:0/2791389381 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:52:28.106 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.424+0000 7f8596ffd700 1 -- 172.21.15.17:0/2791389381 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8588009550 con 0x7f859807be00 2023-07-14T11:52:28.107 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.424+0000 7f8596ffd700 1 --2- 172.21.15.17:0/2791389381 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f859807be00 0x7f859819b340 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8588009fb0 tx=0x7f8588011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:28.107 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.425+0000 7f857ffff700 1 -- 172.21.15.17:0/2791389381 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8588003a40 con 0x7f859807be00 2023-07-14T11:52:28.107 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.425+0000 7f857ffff700 1 -- 172.21.15.17:0/2791389381 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8588003ba0 con 0x7f859807be00 2023-07-14T11:52:28.107 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.425+0000 7f857ffff700 1 -- 172.21.15.17:0/2791389381 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f858801a750 con 0x7f859807be00 2023-07-14T11:52:28.107 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.425+0000 7f859d7c8700 1 -- 172.21.15.17:0/2791389381 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f859819ba80 con 0x7f859807be00 2023-07-14T11:52:28.107 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.425+0000 7f859d7c8700 1 -- 172.21.15.17:0/2791389381 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f859819bea0 con 0x7f859807be00 2023-07-14T11:52:28.107 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.426+0000 7f857ffff700 1 -- 172.21.15.17:0/2791389381 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7f8588024b60 con 0x7f859807be00 2023-07-14T11:52:28.107 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.426+0000 7f859d7c8700 1 -- 172.21.15.17:0/2791389381 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85981952f0 con 0x7f859807be00 2023-07-14T11:52:28.108 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.426+0000 7f857ffff700 1 --2- 172.21.15.17:0/2791389381 >> [v2:172.21.15.17:6800/2804227742,v1:172.21.15.17:6801/2804227742] conn(0x7f85800386c0 0x7f858003ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:28.108 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.426+0000 7f857ffff700 1 -- 172.21.15.17:0/2791389381 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f858804c2c0 con 0x7f859807be00 2023-07-14T11:52:28.108 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.427+0000 7f85967fc700 1 --2- 172.21.15.17:0/2791389381 >> [v2:172.21.15.17:6800/2804227742,v1:172.21.15.17:6801/2804227742] conn(0x7f85800386c0 0x7f858003ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:28.108 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.427+0000 7f85967fc700 1 --2- 172.21.15.17:0/2791389381 >> [v2:172.21.15.17:6800/2804227742,v1:172.21.15.17:6801/2804227742] conn(0x7f85800386c0 0x7f858003ab70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f858c006fd0 tx=0x7f858c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:28.108 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.431+0000 7f857ffff700 1 -- 172.21.15.17:0/2791389381 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8588024430 con 0x7f859807be00 2023-07-14T11:52:28.108 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.635+0000 7f859d7c8700 1 -- 172.21.15.17:0/2791389381 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f8598061b40 con 0x7f859807be00 2023-07-14T11:52:28.108 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.636+0000 7f857ffff700 1 -- 172.21.15.17:0/2791389381 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f85880187c0 con 0x7f859807be00 2023-07-14T11:52:28.108 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.639+0000 7f859d7c8700 1 -- 172.21.15.17:0/2791389381 >> [v2:172.21.15.17:6800/2804227742,v1:172.21.15.17:6801/2804227742] conn(0x7f85800386c0 msgr2=0x7f858003ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:28.109 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.639+0000 7f859d7c8700 1 --2- 172.21.15.17:0/2791389381 >> [v2:172.21.15.17:6800/2804227742,v1:172.21.15.17:6801/2804227742] conn(0x7f85800386c0 0x7f858003ab70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f858c006fd0 tx=0x7f858c006e40 comp rx=0 tx=0).stop 2023-07-14T11:52:28.109 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.639+0000 7f859d7c8700 1 -- 172.21.15.17:0/2791389381 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f859807be00 msgr2=0x7f859819b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:28.109 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.639+0000 7f859d7c8700 1 --2- 172.21.15.17:0/2791389381 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f859807be00 0x7f859819b340 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8588009fb0 tx=0x7f8588011ad0 comp rx=0 tx=0).stop 2023-07-14T11:52:28.109 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.639+0000 7f859d7c8700 1 -- 172.21.15.17:0/2791389381 shutdown_connections 2023-07-14T11:52:28.109 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.639+0000 7f859d7c8700 1 --2- 172.21.15.17:0/2791389381 >> [v2:172.21.15.17:6800/2804227742,v1:172.21.15.17:6801/2804227742] conn(0x7f85800386c0 0x7f858003ab70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:28.109 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.639+0000 7f859d7c8700 1 --2- 172.21.15.17:0/2791389381 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f859807be00 0x7f859819b340 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:28.109 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.639+0000 7f859d7c8700 1 -- 172.21.15.17:0/2791389381 >> 172.21.15.17:0/2791389381 conn(0x7f8598100970 msgr2=0x7f85981020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:28.110 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.639+0000 7f859d7c8700 1 -- 172.21.15.17:0/2791389381 shutdown_connections 2023-07-14T11:52:28.110 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:27.639+0000 7f859d7c8700 1 -- 172.21.15.17:0/2791389381 wait complete. 2023-07-14T11:52:28.110 INFO:teuthology.orchestra.run.smithi017.stdout:Enabling cephadm module... 2023-07-14T11:52:28.383 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:28 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/2791389381' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-07-14T11:52:29.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:29 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/2770155744' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-07-14T11:52:29.398 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:29 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: ignoring --setuser ceph since I am not root 2023-07-14T11:52:29.398 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:29 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: ignoring --setgroup ceph since I am not root 2023-07-14T11:52:29.582 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:28.681+0000 7f95e9201700 1 Processor -- start 2023-07-14T11:52:29.582 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:28.681+0000 7f95e9201700 1 -- start start 2023-07-14T11:52:29.583 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:28.681+0000 7f95e9201700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95e4105320 0x7f95e4105730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:29.583 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:28.681+0000 7f95e9201700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e4105c70 con 0x7f95e4105320 2023-07-14T11:52:29.583 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:28.682+0000 7f95e2d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95e4105320 0x7f95e4105730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:29.586 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:28.682+0000 7f95e2d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95e4105320 0x7f95e4105730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53362/0 (socket says 172.21.15.17:53362) 2023-07-14T11:52:29.586 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:28.682+0000 7f95e2d9d700 1 -- 172.21.15.17:0/434305685 learned_addr learned my addr 172.21.15.17:0/434305685 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:52:29.586 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:28.682+0000 7f95e2d9d700 1 -- 172.21.15.17:0/434305685 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e4105db0 con 0x7f95e4105320 2023-07-14T11:52:29.586 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:28.682+0000 7f95e2d9d700 1 --2- 172.21.15.17:0/434305685 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95e4105320 0x7f95e4105730 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f95d40098a0 tx=0x7f95d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a9bef90fcacdd0c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:29.586 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:28.683+0000 7f95e1d9b700 1 -- 172.21.15.17:0/434305685 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f95d4004160 con 0x7f95e4105320 2023-07-14T11:52:29.587 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:28.683+0000 7f95e1d9b700 1 -- 172.21.15.17:0/434305685 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f95d40042c0 con 0x7f95e4105320 2023-07-14T11:52:29.587 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:28.683+0000 7f95e1d9b700 1 -- 172.21.15.17:0/434305685 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f95d4011a00 con 0x7f95e4105320 2023-07-14T11:52:29.587 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:28.683+0000 7f95e9201700 1 -- 172.21.15.17:0/434305685 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95e4105320 msgr2=0x7f95e4105730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:29.587 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:28.683+0000 7f95e9201700 1 --2- 172.21.15.17:0/434305685 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95e4105320 0x7f95e4105730 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f95d40098a0 tx=0x7f95d4009bb0 comp rx=0 tx=0).stop 2023-07-14T11:52:29.587 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:28.683+0000 7f95e9201700 1 -- 172.21.15.17:0/434305685 shutdown_connections 2023-07-14T11:52:29.587 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:28.683+0000 7f95e9201700 1 --2- 172.21.15.17:0/434305685 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95e4105320 0x7f95e4105730 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:29.588 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:28.683+0000 7f95e9201700 1 -- 172.21.15.17:0/434305685 >> 172.21.15.17:0/434305685 conn(0x7f95e4100970 msgr2=0x7f95e4102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:29.588 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:28.683+0000 7f95e9201700 1 -- 172.21.15.17:0/434305685 shutdown_connections 2023-07-14T11:52:29.588 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:28.684+0000 7f95e9201700 1 -- 172.21.15.17:0/434305685 wait complete. 2023-07-14T11:52:29.588 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:28.685+0000 7f95e9201700 1 Processor -- start 2023-07-14T11:52:29.588 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:28.685+0000 7f95e9201700 1 -- start start 2023-07-14T11:52:29.588 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:28.686+0000 7f95e9201700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95e4105320 0x7f95e407bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:29.588 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:28.686+0000 7f95e9201700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e407c1b0 con 0x7f95e4105320 2023-07-14T11:52:29.589 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:28.686+0000 7f95e2d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95e4105320 0x7f95e407bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:29.589 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:28.686+0000 7f95e2d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95e4105320 0x7f95e407bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53368/0 (socket says 172.21.15.17:53368) 2023-07-14T11:52:29.589 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:28.686+0000 7f95e2d9d700 1 -- 172.21.15.17:0/2770155744 learned_addr learned my addr 172.21.15.17:0/2770155744 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:52:29.589 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:28.686+0000 7f95e2d9d700 1 -- 172.21.15.17:0/2770155744 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d4009550 con 0x7f95e4105320 2023-07-14T11:52:29.589 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:28.687+0000 7f95e2d9d700 1 --2- 172.21.15.17:0/2770155744 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95e4105320 0x7f95e407bc70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f95d4009fb0 tx=0x7f95d4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:29.589 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:28.687+0000 7f95cbfff700 1 -- 172.21.15.17:0/2770155744 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f95d4003a40 con 0x7f95e4105320 2023-07-14T11:52:29.589 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:28.687+0000 7f95cbfff700 1 -- 172.21.15.17:0/2770155744 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f95d4003ba0 con 0x7f95e4105320 2023-07-14T11:52:29.589 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:28.687+0000 7f95cbfff700 1 -- 172.21.15.17:0/2770155744 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f95d401a750 con 0x7f95e4105320 2023-07-14T11:52:29.590 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:28.687+0000 7f95e9201700 1 -- 172.21.15.17:0/2770155744 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95e407a2c0 con 0x7f95e4105320 2023-07-14T11:52:29.590 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:28.687+0000 7f95e9201700 1 -- 172.21.15.17:0/2770155744 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e407a6e0 con 0x7f95e4105320 2023-07-14T11:52:29.590 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:28.688+0000 7f95cbfff700 1 -- 172.21.15.17:0/2770155744 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7f95d4024b60 con 0x7f95e4105320 2023-07-14T11:52:29.590 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:28.688+0000 7f95e9201700 1 -- 172.21.15.17:0/2770155744 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95e4061b40 con 0x7f95e4105320 2023-07-14T11:52:29.590 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:28.688+0000 7f95cbfff700 1 --2- 172.21.15.17:0/2770155744 >> [v2:172.21.15.17:6800/2804227742,v1:172.21.15.17:6801/2804227742] conn(0x7f95cc0386c0 0x7f95cc03ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:29.590 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:28.688+0000 7f95cbfff700 1 -- 172.21.15.17:0/2770155744 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f95d404c330 con 0x7f95e4105320 2023-07-14T11:52:29.590 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:28.689+0000 7f95e259c700 1 --2- 172.21.15.17:0/2770155744 >> [v2:172.21.15.17:6800/2804227742,v1:172.21.15.17:6801/2804227742] conn(0x7f95cc0386c0 0x7f95cc03ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:29.592 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:28.689+0000 7f95e259c700 1 --2- 172.21.15.17:0/2770155744 >> [v2:172.21.15.17:6800/2804227742,v1:172.21.15.17:6801/2804227742] conn(0x7f95cc0386c0 0x7f95cc03ab70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f95d8006fd0 tx=0x7f95d8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:29.592 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:28.694+0000 7f95cbfff700 1 -- 172.21.15.17:0/2770155744 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f95d4024430 con 0x7f95e4105320 2023-07-14T11:52:29.592 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:28.957+0000 7f95e9201700 1 -- 172.21.15.17:0/2770155744 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f95e41071e0 con 0x7f95e4105320 2023-07-14T11:52:29.592 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:29.144+0000 7f95cbfff700 1 -- 172.21.15.17:0/2770155744 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f95d402d770 con 0x7f95e4105320 2023-07-14T11:52:29.592 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:29.146+0000 7f95cbfff700 1 -- 172.21.15.17:0/2770155744 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f95d404e710 con 0x7f95e4105320 2023-07-14T11:52:29.592 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:29.148+0000 7f95e9201700 1 -- 172.21.15.17:0/2770155744 >> [v2:172.21.15.17:6800/2804227742,v1:172.21.15.17:6801/2804227742] conn(0x7f95cc0386c0 msgr2=0x7f95cc03ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:29.593 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:29.148+0000 7f95e9201700 1 --2- 172.21.15.17:0/2770155744 >> [v2:172.21.15.17:6800/2804227742,v1:172.21.15.17:6801/2804227742] conn(0x7f95cc0386c0 0x7f95cc03ab70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f95d8006fd0 tx=0x7f95d8006e40 comp rx=0 tx=0).stop 2023-07-14T11:52:29.593 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:29.148+0000 7f95e9201700 1 -- 172.21.15.17:0/2770155744 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95e4105320 msgr2=0x7f95e407bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:29.593 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:29.148+0000 7f95e9201700 1 --2- 172.21.15.17:0/2770155744 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95e4105320 0x7f95e407bc70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f95d4009fb0 tx=0x7f95d4011ad0 comp rx=0 tx=0).stop 2023-07-14T11:52:29.593 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:29.148+0000 7f95e9201700 1 -- 172.21.15.17:0/2770155744 shutdown_connections 2023-07-14T11:52:29.593 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:29.148+0000 7f95e9201700 1 --2- 172.21.15.17:0/2770155744 >> [v2:172.21.15.17:6800/2804227742,v1:172.21.15.17:6801/2804227742] conn(0x7f95cc0386c0 0x7f95cc03ab70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:29.593 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:29.148+0000 7f95e9201700 1 --2- 172.21.15.17:0/2770155744 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95e4105320 0x7f95e407bc70 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:29.593 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:29.149+0000 7f95e9201700 1 -- 172.21.15.17:0/2770155744 >> 172.21.15.17:0/2770155744 conn(0x7f95e4100970 msgr2=0x7f95e41020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:29.594 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:29.149+0000 7f95e9201700 1 -- 172.21.15.17:0/2770155744 shutdown_connections 2023-07-14T11:52:29.594 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:29.149+0000 7f95e9201700 1 -- 172.21.15.17:0/2770155744 wait complete. 2023-07-14T11:52:29.765 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:29 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:29.420+0000 7f987908a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-14T11:52:29.765 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:29 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:29.527+0000 7f987908a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-14T11:52:29.766 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:29 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:29.656+0000 7f987908a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-14T11:52:30.126 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:29 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:29.782+0000 7f987908a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-14T11:52:30.127 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:30 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:30.024+0000 7f987908a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-14T11:52:30.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:30 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/2770155744' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-07-14T11:52:30.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:30 smithi017 ceph-mon[126339]: mgrmap e5: x(active, since 5s) 2023-07-14T11:52:30.738 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:30 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:30.460+0000 7f987908a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-14T11:52:30.738 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:30 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:30.580+0000 7f987908a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-14T11:52:31.127 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:30 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:30.754+0000 7f987908a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-14T11:52:31.127 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:30 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:30.892+0000 7f987908a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-14T11:52:31.176 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout { 2023-07-14T11:52:31.176 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-07-14T11:52:31.177 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "available": true, 2023-07-14T11:52:31.177 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-07-14T11:52:31.177 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-07-14T11:52:31.177 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout } 2023-07-14T11:52:31.177 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.232+0000 7f24d5437700 1 Processor -- start 2023-07-14T11:52:31.177 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.232+0000 7f24d5437700 1 -- start start 2023-07-14T11:52:31.177 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.233+0000 7f24d5437700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24d0105320 0x7f24d0105730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:31.178 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.233+0000 7f24d5437700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d0105c70 con 0x7f24d0105320 2023-07-14T11:52:31.178 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.233+0000 7f24ceffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24d0105320 0x7f24d0105730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:31.178 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.233+0000 7f24ceffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24d0105320 0x7f24d0105730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53382/0 (socket says 172.21.15.17:53382) 2023-07-14T11:52:31.178 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.233+0000 7f24ceffd700 1 -- 172.21.15.17:0/590368274 learned_addr learned my addr 172.21.15.17:0/590368274 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:52:31.178 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.233+0000 7f24ceffd700 1 -- 172.21.15.17:0/590368274 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d0105db0 con 0x7f24d0105320 2023-07-14T11:52:31.178 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.234+0000 7f24ceffd700 1 --2- 172.21.15.17:0/590368274 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24d0105320 0x7f24d0105730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f24c00098a0 tx=0x7f24c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7156cc582da9d5f0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:31.178 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.234+0000 7f24cdffb700 1 -- 172.21.15.17:0/590368274 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24c0004160 con 0x7f24d0105320 2023-07-14T11:52:31.179 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.234+0000 7f24cdffb700 1 -- 172.21.15.17:0/590368274 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f24c00042c0 con 0x7f24d0105320 2023-07-14T11:52:31.179 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.234+0000 7f24cdffb700 1 -- 172.21.15.17:0/590368274 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24c0011a00 con 0x7f24d0105320 2023-07-14T11:52:31.179 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.235+0000 7f24d5437700 1 -- 172.21.15.17:0/590368274 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24d0105320 msgr2=0x7f24d0105730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:31.179 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.235+0000 7f24d5437700 1 --2- 172.21.15.17:0/590368274 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24d0105320 0x7f24d0105730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f24c00098a0 tx=0x7f24c0009bb0 comp rx=0 tx=0).stop 2023-07-14T11:52:31.179 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.235+0000 7f24d5437700 1 -- 172.21.15.17:0/590368274 shutdown_connections 2023-07-14T11:52:31.179 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.235+0000 7f24d5437700 1 --2- 172.21.15.17:0/590368274 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24d0105320 0x7f24d0105730 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:31.179 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.235+0000 7f24d5437700 1 -- 172.21.15.17:0/590368274 >> 172.21.15.17:0/590368274 conn(0x7f24d0100970 msgr2=0x7f24d0102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:31.179 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.235+0000 7f24d5437700 1 -- 172.21.15.17:0/590368274 shutdown_connections 2023-07-14T11:52:31.180 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.235+0000 7f24d5437700 1 -- 172.21.15.17:0/590368274 wait complete. 2023-07-14T11:52:31.180 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.237+0000 7f24d5437700 1 Processor -- start 2023-07-14T11:52:31.180 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.237+0000 7f24d5437700 1 -- start start 2023-07-14T11:52:31.180 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.237+0000 7f24d5437700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24d0105320 0x7f24d019b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:31.180 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.237+0000 7f24d5437700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d019b820 con 0x7f24d0105320 2023-07-14T11:52:31.180 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.238+0000 7f24ceffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24d0105320 0x7f24d019b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:31.180 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.238+0000 7f24ceffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24d0105320 0x7f24d019b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53396/0 (socket says 172.21.15.17:53396) 2023-07-14T11:52:31.181 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.238+0000 7f24ceffd700 1 -- 172.21.15.17:0/764211538 learned_addr learned my addr 172.21.15.17:0/764211538 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:52:31.181 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.238+0000 7f24ceffd700 1 -- 172.21.15.17:0/764211538 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c0009550 con 0x7f24d0105320 2023-07-14T11:52:31.181 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.238+0000 7f24ceffd700 1 --2- 172.21.15.17:0/764211538 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24d0105320 0x7f24d019b2e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f24c0009130 tx=0x7f24c0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:31.181 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.239+0000 7f24b7fff700 1 -- 172.21.15.17:0/764211538 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24c0003b70 con 0x7f24d0105320 2023-07-14T11:52:31.181 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.239+0000 7f24b7fff700 1 -- 172.21.15.17:0/764211538 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f24c0003cd0 con 0x7f24d0105320 2023-07-14T11:52:31.181 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.239+0000 7f24d5437700 1 -- 172.21.15.17:0/764211538 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24d019ba20 con 0x7f24d0105320 2023-07-14T11:52:31.181 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.239+0000 7f24b7fff700 1 -- 172.21.15.17:0/764211538 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24c00197d0 con 0x7f24d0105320 2023-07-14T11:52:31.181 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.239+0000 7f24d5437700 1 -- 172.21.15.17:0/764211538 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24d019be40 con 0x7f24d0105320 2023-07-14T11:52:31.182 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.240+0000 7f24b7fff700 1 -- 172.21.15.17:0/764211538 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f24c001abb0 con 0x7f24d0105320 2023-07-14T11:52:31.182 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.240+0000 7f24d5437700 1 -- 172.21.15.17:0/764211538 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24d01097b0 con 0x7f24d0105320 2023-07-14T11:52:31.182 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.240+0000 7f24b7fff700 1 --2- 172.21.15.17:0/764211538 >> [v2:172.21.15.17:6800/2804227742,v1:172.21.15.17:6801/2804227742] conn(0x7f24b8038710 0x7f24b803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:31.182 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.240+0000 7f24b7fff700 1 -- 172.21.15.17:0/764211538 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f24c004c8f0 con 0x7f24d0105320 2023-07-14T11:52:31.182 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.240+0000 7f24ce7fc700 1 -- 172.21.15.17:0/764211538 >> [v2:172.21.15.17:6800/2804227742,v1:172.21.15.17:6801/2804227742] conn(0x7f24b8038710 msgr2=0x7f24b803abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/2804227742 2023-07-14T11:52:31.182 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.241+0000 7f24ce7fc700 1 --2- 172.21.15.17:0/764211538 >> [v2:172.21.15.17:6800/2804227742,v1:172.21.15.17:6801/2804227742] conn(0x7f24b8038710 0x7f24b803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-14T11:52:31.182 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.245+0000 7f24b7fff700 1 -- 172.21.15.17:0/764211538 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f24c001a430 con 0x7f24d0105320 2023-07-14T11:52:31.182 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.442+0000 7f24ce7fc700 1 -- 172.21.15.17:0/764211538 >> [v2:172.21.15.17:6800/2804227742,v1:172.21.15.17:6801/2804227742] conn(0x7f24b8038710 msgr2=0x7f24b803abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/2804227742 2023-07-14T11:52:31.183 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.442+0000 7f24ce7fc700 1 --2- 172.21.15.17:0/764211538 >> [v2:172.21.15.17:6800/2804227742,v1:172.21.15.17:6801/2804227742] conn(0x7f24b8038710 0x7f24b803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-14T11:52:31.183 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.543+0000 7f24d5437700 1 -- 172.21.15.17:0/764211538 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f24d019ef40 con 0x7f24d0105320 2023-07-14T11:52:31.183 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.543+0000 7f24b7fff700 1 -- 172.21.15.17:0/764211538 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f24c002db40 con 0x7f24d0105320 2023-07-14T11:52:31.183 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.546+0000 7f24d5437700 1 -- 172.21.15.17:0/764211538 >> [v2:172.21.15.17:6800/2804227742,v1:172.21.15.17:6801/2804227742] conn(0x7f24b8038710 msgr2=0x7f24b803abc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T11:52:31.183 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.546+0000 7f24d5437700 1 --2- 172.21.15.17:0/764211538 >> [v2:172.21.15.17:6800/2804227742,v1:172.21.15.17:6801/2804227742] conn(0x7f24b8038710 0x7f24b803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:31.183 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.546+0000 7f24d5437700 1 -- 172.21.15.17:0/764211538 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24d0105320 msgr2=0x7f24d019b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:31.183 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.546+0000 7f24d5437700 1 --2- 172.21.15.17:0/764211538 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24d0105320 0x7f24d019b2e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f24c0009130 tx=0x7f24c0011ad0 comp rx=0 tx=0).stop 2023-07-14T11:52:31.184 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.546+0000 7f24d5437700 1 -- 172.21.15.17:0/764211538 shutdown_connections 2023-07-14T11:52:31.184 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.546+0000 7f24d5437700 1 --2- 172.21.15.17:0/764211538 >> [v2:172.21.15.17:6800/2804227742,v1:172.21.15.17:6801/2804227742] conn(0x7f24b8038710 0x7f24b803abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:31.184 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.546+0000 7f24d5437700 1 --2- 172.21.15.17:0/764211538 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24d0105320 0x7f24d019b2e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:31.184 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.546+0000 7f24d5437700 1 -- 172.21.15.17:0/764211538 >> 172.21.15.17:0/764211538 conn(0x7f24d0100970 msgr2=0x7f24d0102080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:31.184 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.546+0000 7f24d5437700 1 -- 172.21.15.17:0/764211538 shutdown_connections 2023-07-14T11:52:31.184 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:30.546+0000 7f24d5437700 1 -- 172.21.15.17:0/764211538 wait complete. 2023-07-14T11:52:31.184 INFO:teuthology.orchestra.run.smithi017.stdout:Waiting for the mgr to restart... 2023-07-14T11:52:31.184 INFO:teuthology.orchestra.run.smithi017.stdout:Waiting for mgr epoch 5... 2023-07-14T11:52:31.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:31 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/764211538' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-07-14T11:52:31.897 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:31 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:31.477+0000 7f987908a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-14T11:52:32.647 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:32 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:32.251+0000 7f987908a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-14T11:52:32.647 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:32 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:32.462+0000 7f987908a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-14T11:52:33.770 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:33 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:33.436+0000 7f987908a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-14T11:52:33.771 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:33 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:33.559+0000 7f987908a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-14T11:52:34.138 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:33 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:33.786+0000 7f987908a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-14T11:52:34.397 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:34 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:34.154+0000 7f987908a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-14T11:52:34.897 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:34 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:34.433+0000 7f987908a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-14T11:52:34.897 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:34 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:34.630+0000 7f987908a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-14T11:52:35.397 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:34 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:34.982+0000 7f987908a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-14T11:52:36.346 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:35 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:35.934+0000 7f987908a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-14T11:52:36.346 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:36 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:36.112+0000 7f987908a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-14T11:52:36.647 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:36 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:36.361+0000 7f987908a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-14T11:52:36.647 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:36 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:36.512+0000 7f987908a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-14T11:52:37.147 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:36 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:36.792+0000 7f987908a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-14T11:52:38.147 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:37 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:37.885+0000 7f987908a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-14T11:52:39.112 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:38 smithi017 ceph-mon[126339]: Active manager daemon x restarted 2023-07-14T11:52:39.112 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:38 smithi017 ceph-mon[126339]: Activating manager daemon x 2023-07-14T11:52:39.112 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:38 smithi017 ceph-mon[126339]: osdmap e2: 0 total, 0 up, 0 in 2023-07-14T11:52:39.113 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:38 smithi017 ceph-mon[126339]: mgrmap e6: x(active, starting, since 0.00412136s) 2023-07-14T11:52:39.113 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:38 smithi017 ceph-mon[126339]: from='mgr.14120 172.21.15.17:0/3183437250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T11:52:39.113 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:38 smithi017 ceph-mon[126339]: from='mgr.14120 172.21.15.17:0/3183437250' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-14T11:52:39.113 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:38 smithi017 ceph-mon[126339]: from='mgr.14120 172.21.15.17:0/3183437250' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-14T11:52:39.113 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:38 smithi017 ceph-mon[126339]: from='mgr.14120 172.21.15.17:0/3183437250' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-14T11:52:39.113 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:38 smithi017 ceph-mon[126339]: from='mgr.14120 172.21.15.17:0/3183437250' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-14T11:52:39.113 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:38 smithi017 ceph-mon[126339]: Manager daemon x is now available 2023-07-14T11:52:39.114 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:38 smithi017 ceph-mon[126339]: Found migration_current of "None". Setting to last migration. 2023-07-14T11:52:39.114 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:38 smithi017 ceph-mon[126339]: from='mgr.14120 172.21.15.17:0/3183437250' entity='mgr.x' 2023-07-14T11:52:39.114 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:38 smithi017 ceph-mon[126339]: from='mgr.14120 172.21.15.17:0/3183437250' entity='mgr.x' 2023-07-14T11:52:39.114 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:38 smithi017 ceph-mon[126339]: from='mgr.14120 172.21.15.17:0/3183437250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:52:39.114 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:38 smithi017 ceph-mon[126339]: from='mgr.14120 172.21.15.17:0/3183437250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:52:39.114 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:38 smithi017 ceph-mon[126339]: from='mgr.14120 172.21.15.17:0/3183437250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:52:39.114 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:38 smithi017 ceph-mon[126339]: from='mgr.14120 172.21.15.17:0/3183437250' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-14T11:52:39.115 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:38 smithi017 ceph-mon[126339]: from='mgr.14120 172.21.15.17:0/3183437250' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-14T11:52:39.115 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:38 smithi017 ceph-mon[126339]: from='mgr.14120 172.21.15.17:0/3183437250' entity='mgr.x' 2023-07-14T11:52:39.115 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:38 smithi017 ceph-mon[126339]: from='mgr.14120 172.21.15.17:0/3183437250' entity='mgr.x' 2023-07-14T11:52:39.381 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout { 2023-07-14T11:52:39.382 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-07-14T11:52:39.382 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "initialized": true 2023-07-14T11:52:39.382 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout } 2023-07-14T11:52:39.382 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:31.822+0000 7f3605748700 1 Processor -- start 2023-07-14T11:52:39.382 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:31.822+0000 7f3605748700 1 -- start start 2023-07-14T11:52:39.382 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:31.823+0000 7f3605748700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3600105320 0x7f3600105730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:39.382 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:31.823+0000 7f3605748700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3600105c70 con 0x7f3600105320 2023-07-14T11:52:39.383 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:31.823+0000 7f35feffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3600105320 0x7f3600105730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:39.383 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:31.823+0000 7f35feffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3600105320 0x7f3600105730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53408/0 (socket says 172.21.15.17:53408) 2023-07-14T11:52:39.383 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:31.823+0000 7f35feffd700 1 -- 172.21.15.17:0/390791973 learned_addr learned my addr 172.21.15.17:0/390791973 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:52:39.383 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:31.824+0000 7f35feffd700 1 -- 172.21.15.17:0/390791973 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3600105db0 con 0x7f3600105320 2023-07-14T11:52:39.383 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:31.824+0000 7f35feffd700 1 --2- 172.21.15.17:0/390791973 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3600105320 0x7f3600105730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f35f00098a0 tx=0x7f35f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12379190b38d599b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:39.383 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:31.824+0000 7f35fdffb700 1 -- 172.21.15.17:0/390791973 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f35f0004160 con 0x7f3600105320 2023-07-14T11:52:39.384 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:31.824+0000 7f35fdffb700 1 -- 172.21.15.17:0/390791973 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f35f00042c0 con 0x7f3600105320 2023-07-14T11:52:39.384 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:31.824+0000 7f35fdffb700 1 -- 172.21.15.17:0/390791973 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f35f0011a00 con 0x7f3600105320 2023-07-14T11:52:39.384 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:31.825+0000 7f3605748700 1 -- 172.21.15.17:0/390791973 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3600105320 msgr2=0x7f3600105730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:39.384 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:31.825+0000 7f3605748700 1 --2- 172.21.15.17:0/390791973 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3600105320 0x7f3600105730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f35f00098a0 tx=0x7f35f0009bb0 comp rx=0 tx=0).stop 2023-07-14T11:52:39.384 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:31.825+0000 7f3605748700 1 -- 172.21.15.17:0/390791973 shutdown_connections 2023-07-14T11:52:39.384 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:31.825+0000 7f3605748700 1 --2- 172.21.15.17:0/390791973 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3600105320 0x7f3600105730 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:39.384 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:31.825+0000 7f3605748700 1 -- 172.21.15.17:0/390791973 >> 172.21.15.17:0/390791973 conn(0x7f3600100970 msgr2=0x7f3600102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:39.384 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:31.825+0000 7f3605748700 1 -- 172.21.15.17:0/390791973 shutdown_connections 2023-07-14T11:52:39.385 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:31.825+0000 7f3605748700 1 -- 172.21.15.17:0/390791973 wait complete. 2023-07-14T11:52:39.385 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:31.826+0000 7f3605748700 1 Processor -- start 2023-07-14T11:52:39.385 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:31.826+0000 7f3605748700 1 -- start start 2023-07-14T11:52:39.385 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:31.826+0000 7f3605748700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3600105320 0x7f36001990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:39.385 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:31.826+0000 7f3605748700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3600199600 con 0x7f3600105320 2023-07-14T11:52:39.385 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:31.827+0000 7f35feffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3600105320 0x7f36001990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:39.385 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:31.827+0000 7f35feffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3600105320 0x7f36001990c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53414/0 (socket says 172.21.15.17:53414) 2023-07-14T11:52:39.386 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:31.827+0000 7f35feffd700 1 -- 172.21.15.17:0/4062762284 learned_addr learned my addr 172.21.15.17:0/4062762284 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:52:39.386 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:31.827+0000 7f35feffd700 1 -- 172.21.15.17:0/4062762284 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f0009550 con 0x7f3600105320 2023-07-14T11:52:39.386 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:31.827+0000 7f35feffd700 1 --2- 172.21.15.17:0/4062762284 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3600105320 0x7f36001990c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f35f0009130 tx=0x7f35f0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:39.386 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:31.828+0000 7f35e7fff700 1 -- 172.21.15.17:0/4062762284 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f35f0003b70 con 0x7f3600105320 2023-07-14T11:52:39.386 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:31.828+0000 7f35e7fff700 1 -- 172.21.15.17:0/4062762284 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f35f0003cd0 con 0x7f3600105320 2023-07-14T11:52:39.386 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:31.828+0000 7f3605748700 1 -- 172.21.15.17:0/4062762284 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3600199800 con 0x7f3600105320 2023-07-14T11:52:39.386 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:31.828+0000 7f35e7fff700 1 -- 172.21.15.17:0/4062762284 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f35f00197d0 con 0x7f3600105320 2023-07-14T11:52:39.386 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:31.828+0000 7f3605748700 1 -- 172.21.15.17:0/4062762284 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3600199c20 con 0x7f3600105320 2023-07-14T11:52:39.387 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:31.829+0000 7f35e7fff700 1 -- 172.21.15.17:0/4062762284 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f35f001abb0 con 0x7f3600105320 2023-07-14T11:52:39.387 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:31.830+0000 7f35e7fff700 1 --2- 172.21.15.17:0/4062762284 >> [v2:172.21.15.17:6800/2804227742,v1:172.21.15.17:6801/2804227742] conn(0x7f35e8038710 0x7f35e803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:39.387 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:31.830+0000 7f35e7fff700 1 -- 172.21.15.17:0/4062762284 --> [v2:172.21.15.17:6800/2804227742,v1:172.21.15.17:6801/2804227742] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f35e803b2d0 con 0x7f35e8038710 2023-07-14T11:52:39.387 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:31.830+0000 7f35e7fff700 1 -- 172.21.15.17:0/4062762284 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f35f004c8f0 con 0x7f3600105320 2023-07-14T11:52:39.387 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:31.830+0000 7f35fe7fc700 1 -- 172.21.15.17:0/4062762284 >> [v2:172.21.15.17:6800/2804227742,v1:172.21.15.17:6801/2804227742] conn(0x7f35e8038710 msgr2=0x7f35e803abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/2804227742 2023-07-14T11:52:39.387 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:31.830+0000 7f35fe7fc700 1 --2- 172.21.15.17:0/4062762284 >> [v2:172.21.15.17:6800/2804227742,v1:172.21.15.17:6801/2804227742] conn(0x7f35e8038710 0x7f35e803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-14T11:52:39.387 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:32.031+0000 7f35fe7fc700 1 -- 172.21.15.17:0/4062762284 >> [v2:172.21.15.17:6800/2804227742,v1:172.21.15.17:6801/2804227742] conn(0x7f35e8038710 msgr2=0x7f35e803abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/2804227742 2023-07-14T11:52:39.387 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:32.031+0000 7f35fe7fc700 1 --2- 172.21.15.17:0/4062762284 >> [v2:172.21.15.17:6800/2804227742,v1:172.21.15.17:6801/2804227742] conn(0x7f35e8038710 0x7f35e803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-14T11:52:39.388 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:32.432+0000 7f35fe7fc700 1 -- 172.21.15.17:0/4062762284 >> [v2:172.21.15.17:6800/2804227742,v1:172.21.15.17:6801/2804227742] conn(0x7f35e8038710 msgr2=0x7f35e803abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/2804227742 2023-07-14T11:52:39.388 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:32.432+0000 7f35fe7fc700 1 --2- 172.21.15.17:0/4062762284 >> [v2:172.21.15.17:6800/2804227742,v1:172.21.15.17:6801/2804227742] conn(0x7f35e8038710 0x7f35e803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-07-14T11:52:39.388 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:33.232+0000 7f35fe7fc700 1 -- 172.21.15.17:0/4062762284 >> [v2:172.21.15.17:6800/2804227742,v1:172.21.15.17:6801/2804227742] conn(0x7f35e8038710 msgr2=0x7f35e803abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/2804227742 2023-07-14T11:52:39.388 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:33.232+0000 7f35fe7fc700 1 --2- 172.21.15.17:0/4062762284 >> [v2:172.21.15.17:6800/2804227742,v1:172.21.15.17:6801/2804227742] conn(0x7f35e8038710 0x7f35e803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-07-14T11:52:39.388 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:34.833+0000 7f35fe7fc700 1 -- 172.21.15.17:0/4062762284 >> [v2:172.21.15.17:6800/2804227742,v1:172.21.15.17:6801/2804227742] conn(0x7f35e8038710 msgr2=0x7f35e803abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/2804227742 2023-07-14T11:52:39.388 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:34.833+0000 7f35fe7fc700 1 --2- 172.21.15.17:0/4062762284 >> [v2:172.21.15.17:6800/2804227742,v1:172.21.15.17:6801/2804227742] conn(0x7f35e8038710 0x7f35e803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-07-14T11:52:39.388 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:37.889+0000 7f35e7fff700 1 -- 172.21.15.17:0/4062762284 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mgrmap(e 6) v1 ==== 45250+0+0 (secure 0 0 0) 0x7f35f0019a10 con 0x7f3600105320 2023-07-14T11:52:39.389 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:37.889+0000 7f35e7fff700 1 -- 172.21.15.17:0/4062762284 >> [v2:172.21.15.17:6800/2804227742,v1:172.21.15.17:6801/2804227742] conn(0x7f35e8038710 msgr2=0x7f35e803abc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T11:52:39.389 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:37.889+0000 7f35e7fff700 1 --2- 172.21.15.17:0/4062762284 >> [v2:172.21.15.17:6800/2804227742,v1:172.21.15.17:6801/2804227742] conn(0x7f35e8038710 0x7f35e803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:39.389 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:38.891+0000 7f35e7fff700 1 -- 172.21.15.17:0/4062762284 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mgrmap(e 7) v1 ==== 45377+0+0 (secure 0 0 0) 0x7f35f0019d90 con 0x7f3600105320 2023-07-14T11:52:39.389 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:38.891+0000 7f35e7fff700 1 --2- 172.21.15.17:0/4062762284 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7f35e803bfe0 0x7f35e803e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:39.389 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:38.891+0000 7f35e7fff700 1 -- 172.21.15.17:0/4062762284 --> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f35e803b2d0 con 0x7f35e803bfe0 2023-07-14T11:52:39.389 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:38.892+0000 7f35fe7fc700 1 --2- 172.21.15.17:0/4062762284 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7f35e803bfe0 0x7f35e803e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:39.389 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:38.893+0000 7f35fe7fc700 1 --2- 172.21.15.17:0/4062762284 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7f35e803bfe0 0x7f35e803e3c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f35f4003a10 tx=0x7f35f4009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:39.389 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:38.894+0000 7f35e7fff700 1 -- 172.21.15.17:0/4062762284 <== mgr.14120 v2:172.21.15.17:6800/1059284716 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f35e803b2d0 con 0x7f35e803bfe0 2023-07-14T11:52:39.390 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:38.900+0000 7f3605748700 1 -- 172.21.15.17:0/4062762284 --> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f360019a400 con 0x7f35e803bfe0 2023-07-14T11:52:39.390 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:38.901+0000 7f35e7fff700 1 -- 172.21.15.17:0/4062762284 <== mgr.14120 v2:172.21.15.17:6800/1059284716 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f360019a400 con 0x7f35e803bfe0 2023-07-14T11:52:39.390 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:38.901+0000 7f3605748700 1 -- 172.21.15.17:0/4062762284 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7f35e803bfe0 msgr2=0x7f35e803e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:39.390 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:38.901+0000 7f3605748700 1 --2- 172.21.15.17:0/4062762284 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7f35e803bfe0 0x7f35e803e3c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f35f4003a10 tx=0x7f35f4009290 comp rx=0 tx=0).stop 2023-07-14T11:52:39.390 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:38.901+0000 7f3605748700 1 -- 172.21.15.17:0/4062762284 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3600105320 msgr2=0x7f36001990c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:39.390 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:38.901+0000 7f3605748700 1 --2- 172.21.15.17:0/4062762284 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3600105320 0x7f36001990c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f35f0009130 tx=0x7f35f0011ad0 comp rx=0 tx=0).stop 2023-07-14T11:52:39.390 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:38.901+0000 7f3605748700 1 -- 172.21.15.17:0/4062762284 shutdown_connections 2023-07-14T11:52:39.390 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:38.901+0000 7f3605748700 1 --2- 172.21.15.17:0/4062762284 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7f35e803bfe0 0x7f35e803e3c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:39.391 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:38.901+0000 7f3605748700 1 --2- 172.21.15.17:0/4062762284 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3600105320 0x7f36001990c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:39.391 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:38.901+0000 7f3605748700 1 --2- 172.21.15.17:0/4062762284 >> [v2:172.21.15.17:6800/2804227742,v1:172.21.15.17:6801/2804227742] conn(0x7f35e8038710 0x7f35e803abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:39.391 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:38.901+0000 7f3605748700 1 -- 172.21.15.17:0/4062762284 >> 172.21.15.17:0/4062762284 conn(0x7f3600100970 msgr2=0x7f36001020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:39.391 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:38.902+0000 7f3605748700 1 -- 172.21.15.17:0/4062762284 shutdown_connections 2023-07-14T11:52:39.391 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:38.902+0000 7f3605748700 1 -- 172.21.15.17:0/4062762284 wait complete. 2023-07-14T11:52:39.391 INFO:teuthology.orchestra.run.smithi017.stdout:mgr epoch 5 is available 2023-07-14T11:52:39.391 INFO:teuthology.orchestra.run.smithi017.stdout:Setting orchestrator backend to cephadm... 2023-07-14T11:52:40.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:39 smithi017 ceph-mon[126339]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-07-14T11:52:40.148 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:39 smithi017 ceph-mon[126339]: mgrmap e7: x(active, since 1.00715s) 2023-07-14T11:52:40.148 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:39 smithi017 ceph-mon[126339]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-07-14T11:52:40.660 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:39.953+0000 7fa0742aa700 1 Processor -- start 2023-07-14T11:52:40.661 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:39.954+0000 7fa0742aa700 1 -- start start 2023-07-14T11:52:40.661 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:39.954+0000 7fa0742aa700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa06c105320 0x7fa06c105730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:40.661 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:39.954+0000 7fa0742aa700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c105c70 con 0x7fa06c105320 2023-07-14T11:52:40.661 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:39.954+0000 7fa072046700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa06c105320 0x7fa06c105730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:40.661 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:39.954+0000 7fa072046700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa06c105320 0x7fa06c105730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47664/0 (socket says 172.21.15.17:47664) 2023-07-14T11:52:40.661 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:39.954+0000 7fa072046700 1 -- 172.21.15.17:0/3749376683 learned_addr learned my addr 172.21.15.17:0/3749376683 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:52:40.661 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:39.955+0000 7fa072046700 1 -- 172.21.15.17:0/3749376683 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa06c105db0 con 0x7fa06c105320 2023-07-14T11:52:40.661 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:39.955+0000 7fa072046700 1 --2- 172.21.15.17:0/3749376683 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa06c105320 0x7fa06c105730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa0680098a0 tx=0x7fa068009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f350e109b88cf69c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:40.662 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:39.955+0000 7fa071044700 1 -- 172.21.15.17:0/3749376683 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa068004160 con 0x7fa06c105320 2023-07-14T11:52:40.662 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:39.955+0000 7fa071044700 1 -- 172.21.15.17:0/3749376683 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa0680042c0 con 0x7fa06c105320 2023-07-14T11:52:40.662 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:39.955+0000 7fa071044700 1 -- 172.21.15.17:0/3749376683 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa068011a00 con 0x7fa06c105320 2023-07-14T11:52:40.662 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:39.956+0000 7fa0742aa700 1 -- 172.21.15.17:0/3749376683 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa06c105320 msgr2=0x7fa06c105730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:40.662 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:39.956+0000 7fa0742aa700 1 --2- 172.21.15.17:0/3749376683 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa06c105320 0x7fa06c105730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa0680098a0 tx=0x7fa068009bb0 comp rx=0 tx=0).stop 2023-07-14T11:52:40.663 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:39.956+0000 7fa0742aa700 1 -- 172.21.15.17:0/3749376683 shutdown_connections 2023-07-14T11:52:40.663 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:39.956+0000 7fa0742aa700 1 --2- 172.21.15.17:0/3749376683 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa06c105320 0x7fa06c105730 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:40.663 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:39.956+0000 7fa0742aa700 1 -- 172.21.15.17:0/3749376683 >> 172.21.15.17:0/3749376683 conn(0x7fa06c100970 msgr2=0x7fa06c102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:40.663 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:39.956+0000 7fa0742aa700 1 -- 172.21.15.17:0/3749376683 shutdown_connections 2023-07-14T11:52:40.663 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:39.956+0000 7fa0742aa700 1 -- 172.21.15.17:0/3749376683 wait complete. 2023-07-14T11:52:40.663 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:39.957+0000 7fa0742aa700 1 Processor -- start 2023-07-14T11:52:40.663 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:39.957+0000 7fa0742aa700 1 -- start start 2023-07-14T11:52:40.664 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:39.957+0000 7fa0742aa700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa06c105320 0x7fa06c19b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:40.664 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:39.957+0000 7fa0742aa700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c19b820 con 0x7fa06c105320 2023-07-14T11:52:40.664 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:39.958+0000 7fa072046700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa06c105320 0x7fa06c19b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:40.664 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:39.958+0000 7fa072046700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa06c105320 0x7fa06c19b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47670/0 (socket says 172.21.15.17:47670) 2023-07-14T11:52:40.664 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:39.958+0000 7fa072046700 1 -- 172.21.15.17:0/2314019546 learned_addr learned my addr 172.21.15.17:0/2314019546 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:52:40.664 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:39.958+0000 7fa072046700 1 -- 172.21.15.17:0/2314019546 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa068009550 con 0x7fa06c105320 2023-07-14T11:52:40.664 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:39.958+0000 7fa072046700 1 --2- 172.21.15.17:0/2314019546 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa06c105320 0x7fa06c19b2e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa068009130 tx=0x7fa068011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:40.665 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:39.959+0000 7fa0637fe700 1 -- 172.21.15.17:0/2314019546 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa068003b70 con 0x7fa06c105320 2023-07-14T11:52:40.665 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:39.959+0000 7fa0742aa700 1 -- 172.21.15.17:0/2314019546 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa06c19ba20 con 0x7fa06c105320 2023-07-14T11:52:40.665 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:39.959+0000 7fa0637fe700 1 -- 172.21.15.17:0/2314019546 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa068003cd0 con 0x7fa06c105320 2023-07-14T11:52:40.665 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:39.959+0000 7fa0637fe700 1 -- 172.21.15.17:0/2314019546 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0680197d0 con 0x7fa06c105320 2023-07-14T11:52:40.665 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:39.959+0000 7fa0742aa700 1 -- 172.21.15.17:0/2314019546 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa06c19be40 con 0x7fa06c105320 2023-07-14T11:52:40.665 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:39.960+0000 7fa0742aa700 1 -- 172.21.15.17:0/2314019546 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa06c1097b0 con 0x7fa06c105320 2023-07-14T11:52:40.665 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:39.960+0000 7fa0637fe700 1 -- 172.21.15.17:0/2314019546 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fa06801abb0 con 0x7fa06c105320 2023-07-14T11:52:40.665 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:39.961+0000 7fa0637fe700 1 --2- 172.21.15.17:0/2314019546 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fa058038710 0x7fa05803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:40.666 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:39.961+0000 7fa0637fe700 1 -- 172.21.15.17:0/2314019546 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa06804c980 con 0x7fa06c105320 2023-07-14T11:52:40.666 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:39.961+0000 7fa071845700 1 --2- 172.21.15.17:0/2314019546 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fa058038710 0x7fa05803abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:40.666 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:39.962+0000 7fa071845700 1 --2- 172.21.15.17:0/2314019546 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fa058038710 0x7fa05803abc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa05c006fd0 tx=0x7fa05c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:40.666 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:39.966+0000 7fa0637fe700 1 -- 172.21.15.17:0/2314019546 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa06801a430 con 0x7fa06c105320 2023-07-14T11:52:40.666 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:40.185+0000 7fa0742aa700 1 -- 172.21.15.17:0/2314019546 --> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fa06c102ca0 con 0x7fa058038710 2023-07-14T11:52:40.666 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:40.191+0000 7fa0637fe700 1 -- 172.21.15.17:0/2314019546 <== mgr.14120 v2:172.21.15.17:6800/1059284716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fa06c102ca0 con 0x7fa058038710 2023-07-14T11:52:40.666 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:40.194+0000 7fa0742aa700 1 -- 172.21.15.17:0/2314019546 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fa058038710 msgr2=0x7fa05803abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:40.666 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:40.194+0000 7fa0742aa700 1 --2- 172.21.15.17:0/2314019546 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fa058038710 0x7fa05803abc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa05c006fd0 tx=0x7fa05c006e40 comp rx=0 tx=0).stop 2023-07-14T11:52:40.667 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:40.194+0000 7fa0742aa700 1 -- 172.21.15.17:0/2314019546 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa06c105320 msgr2=0x7fa06c19b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:40.667 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:40.194+0000 7fa0742aa700 1 --2- 172.21.15.17:0/2314019546 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa06c105320 0x7fa06c19b2e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa068009130 tx=0x7fa068011ad0 comp rx=0 tx=0).stop 2023-07-14T11:52:40.667 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:40.194+0000 7fa0742aa700 1 -- 172.21.15.17:0/2314019546 shutdown_connections 2023-07-14T11:52:40.667 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:40.194+0000 7fa0742aa700 1 --2- 172.21.15.17:0/2314019546 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fa058038710 0x7fa05803abc0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:40.667 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:40.194+0000 7fa0742aa700 1 --2- 172.21.15.17:0/2314019546 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa06c105320 0x7fa06c19b2e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:40.667 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:40.194+0000 7fa0742aa700 1 -- 172.21.15.17:0/2314019546 >> 172.21.15.17:0/2314019546 conn(0x7fa06c100970 msgr2=0x7fa06c102040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:40.667 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:40.194+0000 7fa0742aa700 1 -- 172.21.15.17:0/2314019546 shutdown_connections 2023-07-14T11:52:40.668 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:40.194+0000 7fa0742aa700 1 -- 172.21.15.17:0/2314019546 wait complete. 2023-07-14T11:52:40.989 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:40 smithi017 ceph-mon[126339]: mgrmap e8: x(active, since 2s) 2023-07-14T11:52:40.990 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:40 smithi017 ceph-mon[126339]: [14/Jul/2023:11:52:40] ENGINE Bus STARTING 2023-07-14T11:52:40.990 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:40 smithi017 ceph-mon[126339]: [14/Jul/2023:11:52:40] ENGINE Serving on http://172.21.15.17:8765 2023-07-14T11:52:40.990 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:40 smithi017 ceph-mon[126339]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:52:40.990 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:40 smithi017 ceph-mon[126339]: from='mgr.14120 172.21.15.17:0/3183437250' entity='mgr.x' 2023-07-14T11:52:40.990 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:40 smithi017 ceph-mon[126339]: from='mgr.14120 172.21.15.17:0/3183437250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:52:40.990 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:40 smithi017 ceph-mon[126339]: [14/Jul/2023:11:52:40] ENGINE Serving on https://172.21.15.17:7150 2023-07-14T11:52:40.990 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:40 smithi017 ceph-mon[126339]: [14/Jul/2023:11:52:40] ENGINE Bus STARTED 2023-07-14T11:52:40.991 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:40 smithi017 ceph-mon[126339]: from='mgr.14120 172.21.15.17:0/3183437250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:52:41.942 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout value unchanged 2023-07-14T11:52:41.942 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.269+0000 7fc57ff30700 1 Processor -- start 2023-07-14T11:52:41.942 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.269+0000 7fc57ff30700 1 -- start start 2023-07-14T11:52:41.943 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.269+0000 7fc57ff30700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc578105320 0x7fc578105730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:41.943 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.269+0000 7fc57ff30700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc578105c70 con 0x7fc578105320 2023-07-14T11:52:41.943 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.270+0000 7fc57dccc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc578105320 0x7fc578105730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:41.943 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.270+0000 7fc57dccc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc578105320 0x7fc578105730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47680/0 (socket says 172.21.15.17:47680) 2023-07-14T11:52:41.943 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.270+0000 7fc57dccc700 1 -- 172.21.15.17:0/2288347354 learned_addr learned my addr 172.21.15.17:0/2288347354 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:52:41.943 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.270+0000 7fc57dccc700 1 -- 172.21.15.17:0/2288347354 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc578105db0 con 0x7fc578105320 2023-07-14T11:52:41.943 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.270+0000 7fc57dccc700 1 --2- 172.21.15.17:0/2288347354 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc578105320 0x7fc578105730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc5740098a0 tx=0x7fc574009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca826633440103f4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:41.944 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.270+0000 7fc57ccca700 1 -- 172.21.15.17:0/2288347354 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc574004160 con 0x7fc578105320 2023-07-14T11:52:41.944 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.271+0000 7fc57ccca700 1 -- 172.21.15.17:0/2288347354 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc5740042c0 con 0x7fc578105320 2023-07-14T11:52:41.944 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.271+0000 7fc57ccca700 1 -- 172.21.15.17:0/2288347354 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc574011a00 con 0x7fc578105320 2023-07-14T11:52:41.944 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.271+0000 7fc57ff30700 1 -- 172.21.15.17:0/2288347354 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc578105320 msgr2=0x7fc578105730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:41.944 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.271+0000 7fc57ff30700 1 --2- 172.21.15.17:0/2288347354 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc578105320 0x7fc578105730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc5740098a0 tx=0x7fc574009bb0 comp rx=0 tx=0).stop 2023-07-14T11:52:41.944 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.271+0000 7fc57ff30700 1 -- 172.21.15.17:0/2288347354 shutdown_connections 2023-07-14T11:52:41.944 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.271+0000 7fc57ff30700 1 --2- 172.21.15.17:0/2288347354 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc578105320 0x7fc578105730 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:41.945 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.271+0000 7fc57ff30700 1 -- 172.21.15.17:0/2288347354 >> 172.21.15.17:0/2288347354 conn(0x7fc578100970 msgr2=0x7fc578102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:41.945 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.271+0000 7fc57ff30700 1 -- 172.21.15.17:0/2288347354 shutdown_connections 2023-07-14T11:52:41.945 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.271+0000 7fc57ff30700 1 -- 172.21.15.17:0/2288347354 wait complete. 2023-07-14T11:52:41.945 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.272+0000 7fc57ff30700 1 Processor -- start 2023-07-14T11:52:41.945 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.272+0000 7fc57ff30700 1 -- start start 2023-07-14T11:52:41.945 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.273+0000 7fc57ff30700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc578105320 0x7fc57819b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:41.945 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.273+0000 7fc57ff30700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc57819b820 con 0x7fc578105320 2023-07-14T11:52:41.945 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.273+0000 7fc57dccc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc578105320 0x7fc57819b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:41.946 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.273+0000 7fc57dccc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc578105320 0x7fc57819b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47694/0 (socket says 172.21.15.17:47694) 2023-07-14T11:52:41.946 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.273+0000 7fc57dccc700 1 -- 172.21.15.17:0/814415836 learned_addr learned my addr 172.21.15.17:0/814415836 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:52:41.946 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.273+0000 7fc57dccc700 1 -- 172.21.15.17:0/814415836 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc574009550 con 0x7fc578105320 2023-07-14T11:52:41.946 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.274+0000 7fc57dccc700 1 --2- 172.21.15.17:0/814415836 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc578105320 0x7fc57819b2e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc574009fb0 tx=0x7fc574011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:41.946 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.274+0000 7fc56effd700 1 -- 172.21.15.17:0/814415836 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc574003a40 con 0x7fc578105320 2023-07-14T11:52:41.946 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.274+0000 7fc56effd700 1 -- 172.21.15.17:0/814415836 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc574003ba0 con 0x7fc578105320 2023-07-14T11:52:41.946 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.274+0000 7fc57ff30700 1 -- 172.21.15.17:0/814415836 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc57819ba20 con 0x7fc578105320 2023-07-14T11:52:41.946 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.274+0000 7fc56effd700 1 -- 172.21.15.17:0/814415836 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc57401a750 con 0x7fc578105320 2023-07-14T11:52:41.947 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.274+0000 7fc57ff30700 1 -- 172.21.15.17:0/814415836 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc57819be40 con 0x7fc578105320 2023-07-14T11:52:41.947 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.275+0000 7fc57ff30700 1 -- 172.21.15.17:0/814415836 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5781097b0 con 0x7fc578105320 2023-07-14T11:52:41.947 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.275+0000 7fc56effd700 1 -- 172.21.15.17:0/814415836 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fc574024b60 con 0x7fc578105320 2023-07-14T11:52:41.947 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.276+0000 7fc56effd700 1 --2- 172.21.15.17:0/814415836 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fc5640386c0 0x7fc56403ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:41.947 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.276+0000 7fc56effd700 1 -- 172.21.15.17:0/814415836 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc57404c780 con 0x7fc578105320 2023-07-14T11:52:41.947 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.276+0000 7fc57d4cb700 1 --2- 172.21.15.17:0/814415836 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fc5640386c0 0x7fc56403ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:41.947 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.276+0000 7fc57d4cb700 1 --2- 172.21.15.17:0/814415836 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fc5640386c0 0x7fc56403ab70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc568006fd0 tx=0x7fc568006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:41.948 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.280+0000 7fc56effd700 1 -- 172.21.15.17:0/814415836 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc574018530 con 0x7fc578105320 2023-07-14T11:52:41.948 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.491+0000 7fc57ff30700 1 -- 172.21.15.17:0/814415836 --> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fc578102ca0 con 0x7fc5640386c0 2023-07-14T11:52:41.948 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.492+0000 7fc56effd700 1 -- 172.21.15.17:0/814415836 <== mgr.14120 v2:172.21.15.17:6800/1059284716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fc578102ca0 con 0x7fc5640386c0 2023-07-14T11:52:41.948 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.494+0000 7fc57ff30700 1 -- 172.21.15.17:0/814415836 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fc5640386c0 msgr2=0x7fc56403ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:41.948 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.494+0000 7fc57ff30700 1 --2- 172.21.15.17:0/814415836 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fc5640386c0 0x7fc56403ab70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc568006fd0 tx=0x7fc568006e40 comp rx=0 tx=0).stop 2023-07-14T11:52:41.948 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.495+0000 7fc57ff30700 1 -- 172.21.15.17:0/814415836 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc578105320 msgr2=0x7fc57819b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:41.948 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.495+0000 7fc57ff30700 1 --2- 172.21.15.17:0/814415836 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc578105320 0x7fc57819b2e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc574009fb0 tx=0x7fc574011ad0 comp rx=0 tx=0).stop 2023-07-14T11:52:41.948 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.495+0000 7fc57ff30700 1 -- 172.21.15.17:0/814415836 shutdown_connections 2023-07-14T11:52:41.949 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.495+0000 7fc57ff30700 1 --2- 172.21.15.17:0/814415836 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fc5640386c0 0x7fc56403ab70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:41.949 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.495+0000 7fc57ff30700 1 --2- 172.21.15.17:0/814415836 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc578105320 0x7fc57819b2e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:41.949 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.495+0000 7fc57ff30700 1 -- 172.21.15.17:0/814415836 >> 172.21.15.17:0/814415836 conn(0x7fc578100970 msgr2=0x7fc578102040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:41.949 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.495+0000 7fc57ff30700 1 -- 172.21.15.17:0/814415836 shutdown_connections 2023-07-14T11:52:41.949 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:41.495+0000 7fc57ff30700 1 -- 172.21.15.17:0/814415836 wait complete. 2023-07-14T11:52:41.949 INFO:teuthology.orchestra.run.smithi017.stdout:Generating ssh key... 2023-07-14T11:52:42.647 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:42 smithi017 ceph-mon[126339]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:52:43.147 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:42 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: Generating public/private rsa key pair. 2023-07-14T11:52:43.148 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:42 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: Your identification has been saved in /tmp/tmpuf1hadpu/key. 2023-07-14T11:52:43.148 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:42 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: Your public key has been saved in /tmp/tmpuf1hadpu/key.pub. 2023-07-14T11:52:43.148 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:42 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: The key fingerprint is: 2023-07-14T11:52:43.148 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:42 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: SHA256:kc7cc0QFXSU8os1bIqSYv+D9Nx2g6YjGfZK0bqsDWMo ceph-b3c386cc-223c-11ee-9b33-001a4aab830c 2023-07-14T11:52:43.148 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:42 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: The key's randomart image is: 2023-07-14T11:52:43.148 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:42 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: +---[RSA 3072]----+ 2023-07-14T11:52:43.149 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:42 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: | o=o.+| 2023-07-14T11:52:43.149 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:42 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: | .... +. | 2023-07-14T11:52:43.149 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:42 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: | ooo +.. . | 2023-07-14T11:52:43.149 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:42 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: | . o+.oo.= . | 2023-07-14T11:52:43.149 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:42 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: | . + .S o+.= | 2023-07-14T11:52:43.149 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:42 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: | E . . o oo. . | 2023-07-14T11:52:43.149 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:42 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: | + * * . . | 2023-07-14T11:52:43.150 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:42 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: | * X o o . | 2023-07-14T11:52:43.150 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:42 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: | ..=o=.. . | 2023-07-14T11:52:43.150 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:42 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: +----[SHA256]-----+ 2023-07-14T11:52:43.322 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.527+0000 7fb4967ae700 1 Processor -- start 2023-07-14T11:52:43.322 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.527+0000 7fb4967ae700 1 -- start start 2023-07-14T11:52:43.322 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.527+0000 7fb4967ae700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb49007be40 0x7fb49007a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:43.323 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.527+0000 7fb4967ae700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49007a880 con 0x7fb49007be40 2023-07-14T11:52:43.323 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.527+0000 7fb48ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb49007be40 0x7fb49007a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:43.323 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.527+0000 7fb48ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb49007be40 0x7fb49007a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47698/0 (socket says 172.21.15.17:47698) 2023-07-14T11:52:43.323 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.528+0000 7fb48ffff700 1 -- 172.21.15.17:0/1933769893 learned_addr learned my addr 172.21.15.17:0/1933769893 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:52:43.323 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.528+0000 7fb48ffff700 1 -- 172.21.15.17:0/1933769893 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb49007a9c0 con 0x7fb49007be40 2023-07-14T11:52:43.323 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.528+0000 7fb48ffff700 1 --2- 172.21.15.17:0/1933769893 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb49007be40 0x7fb49007a340 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb4800098a0 tx=0x7fb480009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ce8a796e9041c8c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:43.323 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.528+0000 7fb48effd700 1 -- 172.21.15.17:0/1933769893 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb480004160 con 0x7fb49007be40 2023-07-14T11:52:43.324 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.528+0000 7fb48effd700 1 -- 172.21.15.17:0/1933769893 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb4800042c0 con 0x7fb49007be40 2023-07-14T11:52:43.324 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.528+0000 7fb48effd700 1 -- 172.21.15.17:0/1933769893 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb480011a00 con 0x7fb49007be40 2023-07-14T11:52:43.324 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.529+0000 7fb4967ae700 1 -- 172.21.15.17:0/1933769893 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb49007be40 msgr2=0x7fb49007a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:43.324 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.529+0000 7fb4967ae700 1 --2- 172.21.15.17:0/1933769893 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb49007be40 0x7fb49007a340 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb4800098a0 tx=0x7fb480009bb0 comp rx=0 tx=0).stop 2023-07-14T11:52:43.324 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.529+0000 7fb4967ae700 1 -- 172.21.15.17:0/1933769893 shutdown_connections 2023-07-14T11:52:43.324 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.529+0000 7fb4967ae700 1 --2- 172.21.15.17:0/1933769893 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb49007be40 0x7fb49007a340 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:43.324 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.529+0000 7fb4967ae700 1 -- 172.21.15.17:0/1933769893 >> 172.21.15.17:0/1933769893 conn(0x7fb490100970 msgr2=0x7fb490102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:43.325 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.529+0000 7fb4967ae700 1 -- 172.21.15.17:0/1933769893 shutdown_connections 2023-07-14T11:52:43.325 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.529+0000 7fb4967ae700 1 -- 172.21.15.17:0/1933769893 wait complete. 2023-07-14T11:52:43.325 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.530+0000 7fb4967ae700 1 Processor -- start 2023-07-14T11:52:43.325 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.530+0000 7fb4967ae700 1 -- start start 2023-07-14T11:52:43.325 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.531+0000 7fb4967ae700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb49007be40 0x7fb49019b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:43.325 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.531+0000 7fb4967ae700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49019b860 con 0x7fb49007be40 2023-07-14T11:52:43.325 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.531+0000 7fb48ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb49007be40 0x7fb49019b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:43.325 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.531+0000 7fb48ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb49007be40 0x7fb49019b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47710/0 (socket says 172.21.15.17:47710) 2023-07-14T11:52:43.326 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.531+0000 7fb48ffff700 1 -- 172.21.15.17:0/1588410735 learned_addr learned my addr 172.21.15.17:0/1588410735 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:52:43.326 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.531+0000 7fb48ffff700 1 -- 172.21.15.17:0/1588410735 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb480009550 con 0x7fb49007be40 2023-07-14T11:52:43.326 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.532+0000 7fb48ffff700 1 --2- 172.21.15.17:0/1588410735 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb49007be40 0x7fb49019b320 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb480009130 tx=0x7fb480011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:43.326 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.532+0000 7fb48d7fa700 1 -- 172.21.15.17:0/1588410735 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb480003ab0 con 0x7fb49007be40 2023-07-14T11:52:43.326 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.532+0000 7fb48d7fa700 1 -- 172.21.15.17:0/1588410735 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb480003c10 con 0x7fb49007be40 2023-07-14T11:52:43.326 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.532+0000 7fb4967ae700 1 -- 172.21.15.17:0/1588410735 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb49019ba60 con 0x7fb49007be40 2023-07-14T11:52:43.326 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.532+0000 7fb48d7fa700 1 -- 172.21.15.17:0/1588410735 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4800247d0 con 0x7fb49007be40 2023-07-14T11:52:43.327 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.532+0000 7fb4967ae700 1 -- 172.21.15.17:0/1588410735 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb49019be80 con 0x7fb49007be40 2023-07-14T11:52:43.327 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.533+0000 7fb48d7fa700 1 -- 172.21.15.17:0/1588410735 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fb48001abb0 con 0x7fb49007be40 2023-07-14T11:52:43.327 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.533+0000 7fb4967ae700 1 -- 172.21.15.17:0/1588410735 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb490195300 con 0x7fb49007be40 2023-07-14T11:52:43.327 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.534+0000 7fb48d7fa700 1 --2- 172.21.15.17:0/1588410735 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fb478038760 0x7fb47803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:43.327 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.534+0000 7fb48d7fa700 1 -- 172.21.15.17:0/1588410735 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb48004c880 con 0x7fb49007be40 2023-07-14T11:52:43.327 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.534+0000 7fb48f7fe700 1 --2- 172.21.15.17:0/1588410735 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fb478038760 0x7fb47803ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:43.327 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.534+0000 7fb48f7fe700 1 --2- 172.21.15.17:0/1588410735 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fb478038760 0x7fb47803ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb484006fd0 tx=0x7fb484006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:43.328 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.539+0000 7fb48d7fa700 1 -- 172.21.15.17:0/1588410735 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb480018920 con 0x7fb49007be40 2023-07-14T11:52:43.328 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.745+0000 7fb4967ae700 1 -- 172.21.15.17:0/1588410735 --> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fb490060950 con 0x7fb478038760 2023-07-14T11:52:43.328 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.830+0000 7fb48d7fa700 1 -- 172.21.15.17:0/1588410735 <== mgr.14120 v2:172.21.15.17:6800/1059284716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fb490060950 con 0x7fb478038760 2023-07-14T11:52:43.328 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.833+0000 7fb4967ae700 1 -- 172.21.15.17:0/1588410735 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fb478038760 msgr2=0x7fb47803ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:43.328 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.833+0000 7fb4967ae700 1 --2- 172.21.15.17:0/1588410735 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fb478038760 0x7fb47803ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb484006fd0 tx=0x7fb484006e40 comp rx=0 tx=0).stop 2023-07-14T11:52:43.328 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.833+0000 7fb4967ae700 1 -- 172.21.15.17:0/1588410735 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb49007be40 msgr2=0x7fb49019b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:43.328 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.833+0000 7fb4967ae700 1 --2- 172.21.15.17:0/1588410735 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb49007be40 0x7fb49019b320 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb480009130 tx=0x7fb480011ad0 comp rx=0 tx=0).stop 2023-07-14T11:52:43.328 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.833+0000 7fb4967ae700 1 -- 172.21.15.17:0/1588410735 shutdown_connections 2023-07-14T11:52:43.329 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.833+0000 7fb4967ae700 1 --2- 172.21.15.17:0/1588410735 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fb478038760 0x7fb47803ac10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:43.329 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.833+0000 7fb4967ae700 1 --2- 172.21.15.17:0/1588410735 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb49007be40 0x7fb49019b320 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:43.329 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.833+0000 7fb4967ae700 1 -- 172.21.15.17:0/1588410735 >> 172.21.15.17:0/1588410735 conn(0x7fb490100970 msgr2=0x7fb490102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:43.329 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.833+0000 7fb4967ae700 1 -- 172.21.15.17:0/1588410735 shutdown_connections 2023-07-14T11:52:43.329 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:42.833+0000 7fb4967ae700 1 -- 172.21.15.17:0/1588410735 wait complete. 2023-07-14T11:52:43.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:43 smithi017 ceph-mon[126339]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:52:43.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:43 smithi017 ceph-mon[126339]: Generating ssh key... 2023-07-14T11:52:43.898 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:43 smithi017 ceph-mon[126339]: from='mgr.14120 172.21.15.17:0/3183437250' entity='mgr.x' 2023-07-14T11:52:43.898 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:43 smithi017 ceph-mon[126339]: from='mgr.14120 172.21.15.17:0/3183437250' entity='mgr.x' 2023-07-14T11:52:44.600 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHEDJqnq89C2tfGNhua+JBX8WLo8suZasySMTCYKhU7YrirHfsweZzISghGTSXhEHSWZKTebQ2Frr1nZjlr8KxyKwVb1etzJVQL16ZBHiEcIawZj9Wbvv8BX6zBcRP3LG1/S4onylYhkWkuow91spLrISSqQL2EdjcRAgepPs0KV39gVRHbVDQ2Y3syjoZ9xGqCGkHg+uJu6fJupm/oD42FHpB7zgq6LKsa/fYLpR6KDbXXFVjMlrQp8sjTtOa+rSuN5LPcXeGpGVgKv9IFHTE01USoyFqFLM80UHCqHJU5wXJxJB368PX6noJpiFoYVI+GV0orxNJiQnrVTlyTzldUQaXcUZ3quPKctmCejVl0cEIl7hlO3z8PV3LucxA42gg/Zuk6qz0qrd82gCP5FXE9yBxzpVMIgZe1hIufu3/5Vk59KmXa5iGZaNbLPB8YKKotSOPFONpGuSpybuPvJvmk1SqXhmVNKgEqo5vRfZdGEt/nmejbcKyKAJXn2hfrvE= ceph-b3c386cc-223c-11ee-9b33-001a4aab830c 2023-07-14T11:52:44.600 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:43.876+0000 7fef4443f700 1 Processor -- start 2023-07-14T11:52:44.601 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:43.876+0000 7fef4443f700 1 -- start start 2023-07-14T11:52:44.601 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:43.877+0000 7fef4443f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef3c07be20 0x7fef3c07a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:44.601 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:43.877+0000 7fef4443f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef3c07a860 con 0x7fef3c07be20 2023-07-14T11:52:44.601 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:43.877+0000 7fef421db700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef3c07be20 0x7fef3c07a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:44.601 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:43.877+0000 7fef421db700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef3c07be20 0x7fef3c07a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42884/0 (socket says 172.21.15.17:42884) 2023-07-14T11:52:44.601 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:43.877+0000 7fef421db700 1 -- 172.21.15.17:0/3709979531 learned_addr learned my addr 172.21.15.17:0/3709979531 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:52:44.601 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:43.878+0000 7fef421db700 1 -- 172.21.15.17:0/3709979531 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef3c07a9a0 con 0x7fef3c07be20 2023-07-14T11:52:44.602 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:43.878+0000 7fef421db700 1 --2- 172.21.15.17:0/3709979531 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef3c07be20 0x7fef3c07a320 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fef380098a0 tx=0x7fef38009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89f16f576bc64ed5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:44.602 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:43.878+0000 7fef411d9700 1 -- 172.21.15.17:0/3709979531 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef38004160 con 0x7fef3c07be20 2023-07-14T11:52:44.602 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:43.878+0000 7fef411d9700 1 -- 172.21.15.17:0/3709979531 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fef380042c0 con 0x7fef3c07be20 2023-07-14T11:52:44.602 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:43.878+0000 7fef411d9700 1 -- 172.21.15.17:0/3709979531 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef38011a00 con 0x7fef3c07be20 2023-07-14T11:52:44.602 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:43.879+0000 7fef4443f700 1 -- 172.21.15.17:0/3709979531 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef3c07be20 msgr2=0x7fef3c07a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:44.605 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:43.879+0000 7fef4443f700 1 --2- 172.21.15.17:0/3709979531 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef3c07be20 0x7fef3c07a320 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fef380098a0 tx=0x7fef38009bb0 comp rx=0 tx=0).stop 2023-07-14T11:52:44.605 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:43.879+0000 7fef4443f700 1 -- 172.21.15.17:0/3709979531 shutdown_connections 2023-07-14T11:52:44.606 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:43.879+0000 7fef4443f700 1 --2- 172.21.15.17:0/3709979531 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef3c07be20 0x7fef3c07a320 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:44.606 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:43.879+0000 7fef4443f700 1 -- 172.21.15.17:0/3709979531 >> 172.21.15.17:0/3709979531 conn(0x7fef3c100970 msgr2=0x7fef3c102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:44.606 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:43.879+0000 7fef4443f700 1 -- 172.21.15.17:0/3709979531 shutdown_connections 2023-07-14T11:52:44.606 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:43.879+0000 7fef4443f700 1 -- 172.21.15.17:0/3709979531 wait complete. 2023-07-14T11:52:44.606 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:43.880+0000 7fef4443f700 1 Processor -- start 2023-07-14T11:52:44.606 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:43.880+0000 7fef4443f700 1 -- start start 2023-07-14T11:52:44.606 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:43.880+0000 7fef4443f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef3c07be20 0x7fef3c19b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:44.607 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:43.880+0000 7fef4443f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef3c19b880 con 0x7fef3c07be20 2023-07-14T11:52:44.607 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:43.880+0000 7fef421db700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef3c07be20 0x7fef3c19b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:44.607 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:43.880+0000 7fef421db700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef3c07be20 0x7fef3c19b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42886/0 (socket says 172.21.15.17:42886) 2023-07-14T11:52:44.607 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:43.880+0000 7fef421db700 1 -- 172.21.15.17:0/1221909626 learned_addr learned my addr 172.21.15.17:0/1221909626 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:52:44.607 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:43.881+0000 7fef421db700 1 -- 172.21.15.17:0/1221909626 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef38009550 con 0x7fef3c07be20 2023-07-14T11:52:44.607 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:43.881+0000 7fef421db700 1 --2- 172.21.15.17:0/1221909626 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef3c07be20 0x7fef3c19b340 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fef38003f50 tx=0x7fef38011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:44.607 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:43.881+0000 7fef337fe700 1 -- 172.21.15.17:0/1221909626 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef38003aa0 con 0x7fef3c07be20 2023-07-14T11:52:44.607 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:43.881+0000 7fef337fe700 1 -- 172.21.15.17:0/1221909626 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fef38003c00 con 0x7fef3c07be20 2023-07-14T11:52:44.608 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:43.881+0000 7fef337fe700 1 -- 172.21.15.17:0/1221909626 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef380247c0 con 0x7fef3c07be20 2023-07-14T11:52:44.608 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:43.881+0000 7fef4443f700 1 -- 172.21.15.17:0/1221909626 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef3c19ba80 con 0x7fef3c07be20 2023-07-14T11:52:44.608 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:43.881+0000 7fef4443f700 1 -- 172.21.15.17:0/1221909626 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef3c19bea0 con 0x7fef3c07be20 2023-07-14T11:52:44.608 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:43.883+0000 7fef4443f700 1 -- 172.21.15.17:0/1221909626 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef3c195320 con 0x7fef3c07be20 2023-07-14T11:52:44.608 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:43.883+0000 7fef337fe700 1 -- 172.21.15.17:0/1221909626 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fef3801ab60 con 0x7fef3c07be20 2023-07-14T11:52:44.608 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:43.883+0000 7fef337fe700 1 --2- 172.21.15.17:0/1221909626 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fef28038760 0x7fef2803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:44.608 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:43.883+0000 7fef337fe700 1 -- 172.21.15.17:0/1221909626 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fef3804c740 con 0x7fef3c07be20 2023-07-14T11:52:44.609 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:43.883+0000 7fef419da700 1 --2- 172.21.15.17:0/1221909626 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fef28038760 0x7fef2803ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:44.609 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:43.884+0000 7fef419da700 1 --2- 172.21.15.17:0/1221909626 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fef28038760 0x7fef2803ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fef2c006fd0 tx=0x7fef2c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:44.609 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:43.888+0000 7fef337fe700 1 -- 172.21.15.17:0/1221909626 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fef38024920 con 0x7fef3c07be20 2023-07-14T11:52:44.609 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:44.097+0000 7fef4443f700 1 -- 172.21.15.17:0/1221909626 --> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fef3c060950 con 0x7fef28038760 2023-07-14T11:52:44.609 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:44.098+0000 7fef337fe700 1 -- 172.21.15.17:0/1221909626 <== mgr.14120 v2:172.21.15.17:6800/1059284716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fef3c060950 con 0x7fef28038760 2023-07-14T11:52:44.609 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:44.100+0000 7fef4443f700 1 -- 172.21.15.17:0/1221909626 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fef28038760 msgr2=0x7fef2803ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:44.609 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:44.100+0000 7fef4443f700 1 --2- 172.21.15.17:0/1221909626 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fef28038760 0x7fef2803ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fef2c006fd0 tx=0x7fef2c006e40 comp rx=0 tx=0).stop 2023-07-14T11:52:44.609 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:44.100+0000 7fef4443f700 1 -- 172.21.15.17:0/1221909626 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef3c07be20 msgr2=0x7fef3c19b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:44.610 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:44.100+0000 7fef4443f700 1 --2- 172.21.15.17:0/1221909626 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef3c07be20 0x7fef3c19b340 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fef38003f50 tx=0x7fef38011ad0 comp rx=0 tx=0).stop 2023-07-14T11:52:44.610 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:44.100+0000 7fef4443f700 1 -- 172.21.15.17:0/1221909626 shutdown_connections 2023-07-14T11:52:44.610 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:44.100+0000 7fef4443f700 1 --2- 172.21.15.17:0/1221909626 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fef28038760 0x7fef2803ac10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:44.610 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:44.100+0000 7fef4443f700 1 --2- 172.21.15.17:0/1221909626 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef3c07be20 0x7fef3c19b340 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:44.610 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:44.100+0000 7fef4443f700 1 -- 172.21.15.17:0/1221909626 >> 172.21.15.17:0/1221909626 conn(0x7fef3c100970 msgr2=0x7fef3c102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:44.610 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:44.101+0000 7fef4443f700 1 -- 172.21.15.17:0/1221909626 shutdown_connections 2023-07-14T11:52:44.610 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:44.101+0000 7fef4443f700 1 -- 172.21.15.17:0/1221909626 wait complete. 2023-07-14T11:52:44.610 INFO:teuthology.orchestra.run.smithi017.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-07-14T11:52:44.611 INFO:teuthology.orchestra.run.smithi017.stdout:Adding key to root@localhost authorized_keys... 2023-07-14T11:52:44.611 INFO:teuthology.orchestra.run.smithi017.stdout:Adding host smithi017... 2023-07-14T11:52:44.913 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:44 smithi017 ceph-mon[126339]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:52:46.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:46 smithi017 ceph-mon[126339]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi017", "addr": "172.21.15.17", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:52:46.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:46 smithi017 ceph-mon[126339]: Deploying cephadm binary to smithi017 2023-07-14T11:52:48.995 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout Added host 'smithi017' with addr '172.21.15.17' 2023-07-14T11:52:48.995 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:45.137+0000 7f5852576700 1 Processor -- start 2023-07-14T11:52:48.995 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:45.137+0000 7f5852576700 1 -- start start 2023-07-14T11:52:48.996 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:45.137+0000 7f5852576700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f584c07be00 0x7f584c07a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:48.996 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:45.138+0000 7f5852576700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584c07a840 con 0x7f584c07be00 2023-07-14T11:52:48.996 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:45.138+0000 7f584bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f584c07be00 0x7f584c07a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:48.996 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:45.138+0000 7f584bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f584c07be00 0x7f584c07a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42900/0 (socket says 172.21.15.17:42900) 2023-07-14T11:52:48.996 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:45.138+0000 7f584bfff700 1 -- 172.21.15.17:0/1317642390 learned_addr learned my addr 172.21.15.17:0/1317642390 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:52:48.996 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:45.138+0000 7f584bfff700 1 -- 172.21.15.17:0/1317642390 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f584c07a980 con 0x7f584c07be00 2023-07-14T11:52:48.996 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:45.138+0000 7f584bfff700 1 --2- 172.21.15.17:0/1317642390 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f584c07be00 0x7f584c07a300 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f583c0098a0 tx=0x7f583c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dea71081d9e737c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:48.997 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:45.139+0000 7f584affd700 1 -- 172.21.15.17:0/1317642390 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f583c004160 con 0x7f584c07be00 2023-07-14T11:52:48.997 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:45.139+0000 7f584affd700 1 -- 172.21.15.17:0/1317642390 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f583c0042c0 con 0x7f584c07be00 2023-07-14T11:52:48.997 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:45.139+0000 7f584affd700 1 -- 172.21.15.17:0/1317642390 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f583c011a00 con 0x7f584c07be00 2023-07-14T11:52:48.997 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:45.139+0000 7f5852576700 1 -- 172.21.15.17:0/1317642390 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f584c07be00 msgr2=0x7f584c07a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:48.997 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:45.139+0000 7f5852576700 1 --2- 172.21.15.17:0/1317642390 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f584c07be00 0x7f584c07a300 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f583c0098a0 tx=0x7f583c009bb0 comp rx=0 tx=0).stop 2023-07-14T11:52:48.997 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:45.139+0000 7f5852576700 1 -- 172.21.15.17:0/1317642390 shutdown_connections 2023-07-14T11:52:48.997 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:45.139+0000 7f5852576700 1 --2- 172.21.15.17:0/1317642390 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f584c07be00 0x7f584c07a300 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:48.998 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:45.139+0000 7f5852576700 1 -- 172.21.15.17:0/1317642390 >> 172.21.15.17:0/1317642390 conn(0x7f584c100970 msgr2=0x7f584c102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:48.998 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:45.139+0000 7f5852576700 1 -- 172.21.15.17:0/1317642390 shutdown_connections 2023-07-14T11:52:48.998 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:45.139+0000 7f5852576700 1 -- 172.21.15.17:0/1317642390 wait complete. 2023-07-14T11:52:48.998 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:45.140+0000 7f5852576700 1 Processor -- start 2023-07-14T11:52:48.998 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:45.140+0000 7f5852576700 1 -- start start 2023-07-14T11:52:48.998 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:45.141+0000 7f5852576700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f584c07be00 0x7f584c19b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:49.001 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:45.141+0000 7f5852576700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584c19b880 con 0x7f584c07be00 2023-07-14T11:52:49.001 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:45.141+0000 7f584bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f584c07be00 0x7f584c19b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:49.001 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:45.141+0000 7f584bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f584c07be00 0x7f584c19b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42908/0 (socket says 172.21.15.17:42908) 2023-07-14T11:52:49.001 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:45.141+0000 7f584bfff700 1 -- 172.21.15.17:0/2839242003 learned_addr learned my addr 172.21.15.17:0/2839242003 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:52:49.001 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:45.141+0000 7f584bfff700 1 -- 172.21.15.17:0/2839242003 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f583c009550 con 0x7f584c07be00 2023-07-14T11:52:49.001 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:45.142+0000 7f584bfff700 1 --2- 172.21.15.17:0/2839242003 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f584c07be00 0x7f584c19b340 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f583c009130 tx=0x7f583c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:49.002 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:45.142+0000 7f58497fa700 1 -- 172.21.15.17:0/2839242003 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f583c003ab0 con 0x7f584c07be00 2023-07-14T11:52:49.002 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:45.142+0000 7f58497fa700 1 -- 172.21.15.17:0/2839242003 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f583c003c10 con 0x7f584c07be00 2023-07-14T11:52:49.002 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:45.142+0000 7f5852576700 1 -- 172.21.15.17:0/2839242003 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f584c19ba80 con 0x7f584c07be00 2023-07-14T11:52:49.002 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:45.142+0000 7f58497fa700 1 -- 172.21.15.17:0/2839242003 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f583c0247d0 con 0x7f584c07be00 2023-07-14T11:52:49.002 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:45.142+0000 7f5852576700 1 -- 172.21.15.17:0/2839242003 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f584c19bea0 con 0x7f584c07be00 2023-07-14T11:52:49.002 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:45.143+0000 7f58497fa700 1 -- 172.21.15.17:0/2839242003 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f583c01a430 con 0x7f584c07be00 2023-07-14T11:52:49.002 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:45.143+0000 7f5852576700 1 -- 172.21.15.17:0/2839242003 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f584c1952f0 con 0x7f584c07be00 2023-07-14T11:52:49.002 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:45.143+0000 7f58497fa700 1 --2- 172.21.15.17:0/2839242003 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7f5834038710 0x7f583403abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:49.003 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:45.144+0000 7f58497fa700 1 -- 172.21.15.17:0/2839242003 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f583c04c8e0 con 0x7f584c07be00 2023-07-14T11:52:49.003 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:45.144+0000 7f584b7fe700 1 --2- 172.21.15.17:0/2839242003 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7f5834038710 0x7f583403abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:49.003 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:45.144+0000 7f584b7fe700 1 --2- 172.21.15.17:0/2839242003 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7f5834038710 0x7f583403abc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5840006fd0 tx=0x7f5840006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:49.003 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:45.148+0000 7f58497fa700 1 -- 172.21.15.17:0/2839242003 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f583c018530 con 0x7f584c07be00 2023-07-14T11:52:49.003 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:45.361+0000 7f5852576700 1 -- 172.21.15.17:0/2839242003 --> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi017", "addr": "172.21.15.17", "target": ["mon-mgr", ""]}) v1 -- 0x7f584c060950 con 0x7f5834038710 2023-07-14T11:52:49.003 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:48.577+0000 7f58497fa700 1 -- 172.21.15.17:0/2839242003 <== mgr.14120 v2:172.21.15.17:6800/1059284716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f584c060950 con 0x7f5834038710 2023-07-14T11:52:49.004 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:48.580+0000 7f5852576700 1 -- 172.21.15.17:0/2839242003 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7f5834038710 msgr2=0x7f583403abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:49.004 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:48.580+0000 7f5852576700 1 --2- 172.21.15.17:0/2839242003 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7f5834038710 0x7f583403abc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5840006fd0 tx=0x7f5840006e40 comp rx=0 tx=0).stop 2023-07-14T11:52:49.004 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:48.580+0000 7f5852576700 1 -- 172.21.15.17:0/2839242003 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f584c07be00 msgr2=0x7f584c19b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:49.004 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:48.580+0000 7f5852576700 1 --2- 172.21.15.17:0/2839242003 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f584c07be00 0x7f584c19b340 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f583c009130 tx=0x7f583c011ad0 comp rx=0 tx=0).stop 2023-07-14T11:52:49.004 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:48.580+0000 7f5852576700 1 -- 172.21.15.17:0/2839242003 shutdown_connections 2023-07-14T11:52:49.004 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:48.580+0000 7f5852576700 1 --2- 172.21.15.17:0/2839242003 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7f5834038710 0x7f583403abc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:49.004 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:48.580+0000 7f5852576700 1 --2- 172.21.15.17:0/2839242003 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f584c07be00 0x7f584c19b340 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:49.005 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:48.580+0000 7f5852576700 1 -- 172.21.15.17:0/2839242003 >> 172.21.15.17:0/2839242003 conn(0x7f584c100970 msgr2=0x7f584c1020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:49.005 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:48.581+0000 7f5852576700 1 -- 172.21.15.17:0/2839242003 shutdown_connections 2023-07-14T11:52:49.005 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:48.581+0000 7f5852576700 1 -- 172.21.15.17:0/2839242003 wait complete. 2023-07-14T11:52:49.005 INFO:teuthology.orchestra.run.smithi017.stdout:Deploying unmanaged mon service... 2023-07-14T11:52:49.898 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:49 smithi017 ceph-mon[126339]: from='mgr.14120 172.21.15.17:0/3183437250' entity='mgr.x' 2023-07-14T11:52:49.898 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:49 smithi017 ceph-mon[126339]: Added host smithi017 2023-07-14T11:52:49.898 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:49 smithi017 ceph-mon[126339]: from='mgr.14120 172.21.15.17:0/3183437250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:52:50.679 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-07-14T11:52:50.679 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:49.864+0000 7fca85262700 1 Processor -- start 2023-07-14T11:52:50.679 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:49.864+0000 7fca85262700 1 -- start start 2023-07-14T11:52:50.679 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:49.864+0000 7fca85262700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca80105320 0x7fca80105730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:50.679 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:49.864+0000 7fca85262700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca80105c70 con 0x7fca80105320 2023-07-14T11:52:50.680 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:49.865+0000 7fca7effd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca80105320 0x7fca80105730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:50.680 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:49.865+0000 7fca7effd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca80105320 0x7fca80105730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42924/0 (socket says 172.21.15.17:42924) 2023-07-14T11:52:50.680 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:49.865+0000 7fca7effd700 1 -- 172.21.15.17:0/306170075 learned_addr learned my addr 172.21.15.17:0/306170075 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:52:50.680 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:49.865+0000 7fca7effd700 1 -- 172.21.15.17:0/306170075 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca80105db0 con 0x7fca80105320 2023-07-14T11:52:50.680 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:49.865+0000 7fca7effd700 1 --2- 172.21.15.17:0/306170075 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca80105320 0x7fca80105730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fca700098a0 tx=0x7fca70009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51d2c1dd00217b69 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:50.680 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:49.866+0000 7fca7dffb700 1 -- 172.21.15.17:0/306170075 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fca70004160 con 0x7fca80105320 2023-07-14T11:52:50.680 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:49.866+0000 7fca7dffb700 1 -- 172.21.15.17:0/306170075 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fca700042c0 con 0x7fca80105320 2023-07-14T11:52:50.681 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:49.866+0000 7fca7dffb700 1 -- 172.21.15.17:0/306170075 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fca70011a00 con 0x7fca80105320 2023-07-14T11:52:50.681 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:49.866+0000 7fca85262700 1 -- 172.21.15.17:0/306170075 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca80105320 msgr2=0x7fca80105730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:50.681 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:49.866+0000 7fca85262700 1 --2- 172.21.15.17:0/306170075 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca80105320 0x7fca80105730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fca700098a0 tx=0x7fca70009bb0 comp rx=0 tx=0).stop 2023-07-14T11:52:50.681 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:49.866+0000 7fca85262700 1 -- 172.21.15.17:0/306170075 shutdown_connections 2023-07-14T11:52:50.681 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:49.866+0000 7fca85262700 1 --2- 172.21.15.17:0/306170075 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca80105320 0x7fca80105730 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:50.681 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:49.866+0000 7fca85262700 1 -- 172.21.15.17:0/306170075 >> 172.21.15.17:0/306170075 conn(0x7fca80100970 msgr2=0x7fca80102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:50.681 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:49.867+0000 7fca85262700 1 -- 172.21.15.17:0/306170075 shutdown_connections 2023-07-14T11:52:50.682 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:49.867+0000 7fca85262700 1 -- 172.21.15.17:0/306170075 wait complete. 2023-07-14T11:52:50.682 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:49.868+0000 7fca85262700 1 Processor -- start 2023-07-14T11:52:50.682 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:49.868+0000 7fca85262700 1 -- start start 2023-07-14T11:52:50.682 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:49.868+0000 7fca85262700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca80105320 0x7fca8019b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:50.682 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:49.868+0000 7fca85262700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8019b820 con 0x7fca80105320 2023-07-14T11:52:50.682 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:49.868+0000 7fca7effd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca80105320 0x7fca8019b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:50.682 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:49.868+0000 7fca7effd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca80105320 0x7fca8019b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42930/0 (socket says 172.21.15.17:42930) 2023-07-14T11:52:50.683 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:49.868+0000 7fca7effd700 1 -- 172.21.15.17:0/1407980144 learned_addr learned my addr 172.21.15.17:0/1407980144 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:52:50.683 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:49.869+0000 7fca7effd700 1 -- 172.21.15.17:0/1407980144 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca70009550 con 0x7fca80105320 2023-07-14T11:52:50.683 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:49.869+0000 7fca7effd700 1 --2- 172.21.15.17:0/1407980144 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca80105320 0x7fca8019b2e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fca70009130 tx=0x7fca70011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:50.683 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:49.869+0000 7fca67fff700 1 -- 172.21.15.17:0/1407980144 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fca70003b70 con 0x7fca80105320 2023-07-14T11:52:50.683 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:49.869+0000 7fca67fff700 1 -- 172.21.15.17:0/1407980144 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fca70003cd0 con 0x7fca80105320 2023-07-14T11:52:50.683 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:49.869+0000 7fca85262700 1 -- 172.21.15.17:0/1407980144 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca8019ba20 con 0x7fca80105320 2023-07-14T11:52:50.683 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:49.869+0000 7fca67fff700 1 -- 172.21.15.17:0/1407980144 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fca700197d0 con 0x7fca80105320 2023-07-14T11:52:50.684 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:49.869+0000 7fca85262700 1 -- 172.21.15.17:0/1407980144 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca8019be40 con 0x7fca80105320 2023-07-14T11:52:50.684 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:49.871+0000 7fca67fff700 1 -- 172.21.15.17:0/1407980144 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fca7001abb0 con 0x7fca80105320 2023-07-14T11:52:50.684 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:49.870+0000 7fca85262700 1 -- 172.21.15.17:0/1407980144 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca80061b40 con 0x7fca80105320 2023-07-14T11:52:50.684 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:49.871+0000 7fca67fff700 1 --2- 172.21.15.17:0/1407980144 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fca68038760 0x7fca6803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:50.684 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:49.871+0000 7fca67fff700 1 -- 172.21.15.17:0/1407980144 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fca7004c9b0 con 0x7fca80105320 2023-07-14T11:52:50.684 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:49.871+0000 7fca7e7fc700 1 --2- 172.21.15.17:0/1407980144 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fca68038760 0x7fca6803ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:50.685 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:49.872+0000 7fca7e7fc700 1 --2- 172.21.15.17:0/1407980144 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fca68038760 0x7fca6803ac10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fca74006fd0 tx=0x7fca74006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:50.685 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:49.876+0000 7fca67fff700 1 -- 172.21.15.17:0/1407980144 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fca7001a430 con 0x7fca80105320 2023-07-14T11:52:50.685 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:50.120+0000 7fca85262700 1 -- 172.21.15.17:0/1407980144 --> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fca8019eca0 con 0x7fca68038760 2023-07-14T11:52:50.685 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:50.126+0000 7fca67fff700 1 -- 172.21.15.17:0/1407980144 <== mgr.14120 v2:172.21.15.17:6800/1059284716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fca8019eca0 con 0x7fca68038760 2023-07-14T11:52:50.685 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:50.129+0000 7fca85262700 1 -- 172.21.15.17:0/1407980144 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fca68038760 msgr2=0x7fca6803ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:50.685 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:50.129+0000 7fca85262700 1 --2- 172.21.15.17:0/1407980144 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fca68038760 0x7fca6803ac10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fca74006fd0 tx=0x7fca74006e40 comp rx=0 tx=0).stop 2023-07-14T11:52:50.685 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:50.129+0000 7fca85262700 1 -- 172.21.15.17:0/1407980144 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca80105320 msgr2=0x7fca8019b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:50.685 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:50.129+0000 7fca85262700 1 --2- 172.21.15.17:0/1407980144 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca80105320 0x7fca8019b2e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fca70009130 tx=0x7fca70011ad0 comp rx=0 tx=0).stop 2023-07-14T11:52:50.686 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:50.129+0000 7fca85262700 1 -- 172.21.15.17:0/1407980144 shutdown_connections 2023-07-14T11:52:50.686 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:50.129+0000 7fca85262700 1 --2- 172.21.15.17:0/1407980144 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fca68038760 0x7fca6803ac10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:50.686 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:50.129+0000 7fca85262700 1 --2- 172.21.15.17:0/1407980144 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca80105320 0x7fca8019b2e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:50.686 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:50.129+0000 7fca85262700 1 -- 172.21.15.17:0/1407980144 >> 172.21.15.17:0/1407980144 conn(0x7fca80100970 msgr2=0x7fca801020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:50.686 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:50.129+0000 7fca85262700 1 -- 172.21.15.17:0/1407980144 shutdown_connections 2023-07-14T11:52:50.686 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:50.129+0000 7fca85262700 1 -- 172.21.15.17:0/1407980144 wait complete. 2023-07-14T11:52:50.686 INFO:teuthology.orchestra.run.smithi017.stdout:Deploying unmanaged mgr service... 2023-07-14T11:52:51.316 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:51 smithi017 ceph-mon[126339]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:52:51.317 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:51 smithi017 ceph-mon[126339]: Saving service mon spec with placement count:5 2023-07-14T11:52:51.317 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:51 smithi017 ceph-mon[126339]: from='mgr.14120 172.21.15.17:0/3183437250' entity='mgr.x' 2023-07-14T11:52:52.472 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-07-14T11:52:52.473 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.733+0000 7fd495a0a700 1 Processor -- start 2023-07-14T11:52:52.473 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.734+0000 7fd495a0a700 1 -- start start 2023-07-14T11:52:52.473 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.734+0000 7fd495a0a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd490105320 0x7fd490105730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:52.473 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.734+0000 7fd495a0a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd490105c70 con 0x7fd490105320 2023-07-14T11:52:52.473 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.735+0000 7fd48effd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd490105320 0x7fd490105730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:52.473 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.735+0000 7fd48effd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd490105320 0x7fd490105730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42936/0 (socket says 172.21.15.17:42936) 2023-07-14T11:52:52.474 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.735+0000 7fd48effd700 1 -- 172.21.15.17:0/2466384039 learned_addr learned my addr 172.21.15.17:0/2466384039 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:52:52.474 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.735+0000 7fd48effd700 1 -- 172.21.15.17:0/2466384039 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd490105db0 con 0x7fd490105320 2023-07-14T11:52:52.474 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.735+0000 7fd48effd700 1 --2- 172.21.15.17:0/2466384039 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd490105320 0x7fd490105730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd4800098a0 tx=0x7fd480009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b0e44c7ff7047 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:52.474 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.736+0000 7fd48dffb700 1 -- 172.21.15.17:0/2466384039 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd480004160 con 0x7fd490105320 2023-07-14T11:52:52.474 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.736+0000 7fd48dffb700 1 -- 172.21.15.17:0/2466384039 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4800042c0 con 0x7fd490105320 2023-07-14T11:52:52.474 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.736+0000 7fd48dffb700 1 -- 172.21.15.17:0/2466384039 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd4800119b0 con 0x7fd490105320 2023-07-14T11:52:52.475 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.736+0000 7fd495a0a700 1 -- 172.21.15.17:0/2466384039 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd490105320 msgr2=0x7fd490105730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:52.475 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.736+0000 7fd495a0a700 1 --2- 172.21.15.17:0/2466384039 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd490105320 0x7fd490105730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd4800098a0 tx=0x7fd480009bb0 comp rx=0 tx=0).stop 2023-07-14T11:52:52.475 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.736+0000 7fd495a0a700 1 -- 172.21.15.17:0/2466384039 shutdown_connections 2023-07-14T11:52:52.475 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.737+0000 7fd495a0a700 1 --2- 172.21.15.17:0/2466384039 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd490105320 0x7fd490105730 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:52.475 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.737+0000 7fd495a0a700 1 -- 172.21.15.17:0/2466384039 >> 172.21.15.17:0/2466384039 conn(0x7fd490100970 msgr2=0x7fd490102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:52.475 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.737+0000 7fd495a0a700 1 -- 172.21.15.17:0/2466384039 shutdown_connections 2023-07-14T11:52:52.475 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.737+0000 7fd495a0a700 1 -- 172.21.15.17:0/2466384039 wait complete. 2023-07-14T11:52:52.476 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.738+0000 7fd495a0a700 1 Processor -- start 2023-07-14T11:52:52.476 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.738+0000 7fd495a0a700 1 -- start start 2023-07-14T11:52:52.476 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.738+0000 7fd495a0a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd490105320 0x7fd4901990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:52.476 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.738+0000 7fd495a0a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd490199620 con 0x7fd490105320 2023-07-14T11:52:52.476 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.739+0000 7fd48effd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd490105320 0x7fd4901990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:52.476 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.739+0000 7fd48effd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd490105320 0x7fd4901990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42946/0 (socket says 172.21.15.17:42946) 2023-07-14T11:52:52.477 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.739+0000 7fd48effd700 1 -- 172.21.15.17:0/2699808110 learned_addr learned my addr 172.21.15.17:0/2699808110 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:52:52.477 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.739+0000 7fd48effd700 1 -- 172.21.15.17:0/2699808110 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd480009550 con 0x7fd490105320 2023-07-14T11:52:52.477 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.739+0000 7fd48effd700 1 --2- 172.21.15.17:0/2699808110 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd490105320 0x7fd4901990e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd480003f00 tx=0x7fd480011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:52.477 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.740+0000 7fd494a08700 1 -- 172.21.15.17:0/2699808110 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd480003b20 con 0x7fd490105320 2023-07-14T11:52:52.477 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.740+0000 7fd494a08700 1 -- 172.21.15.17:0/2699808110 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd480003c80 con 0x7fd490105320 2023-07-14T11:52:52.477 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.740+0000 7fd495a0a700 1 -- 172.21.15.17:0/2699808110 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd490199820 con 0x7fd490105320 2023-07-14T11:52:52.477 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.740+0000 7fd494a08700 1 -- 172.21.15.17:0/2699808110 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd480019770 con 0x7fd490105320 2023-07-14T11:52:52.478 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.740+0000 7fd495a0a700 1 -- 172.21.15.17:0/2699808110 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd490199c40 con 0x7fd490105320 2023-07-14T11:52:52.478 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.741+0000 7fd495a0a700 1 -- 172.21.15.17:0/2699808110 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4901097b0 con 0x7fd490105320 2023-07-14T11:52:52.478 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.741+0000 7fd494a08700 1 -- 172.21.15.17:0/2699808110 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fd48001ab60 con 0x7fd490105320 2023-07-14T11:52:52.478 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.742+0000 7fd494a08700 1 --2- 172.21.15.17:0/2699808110 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fd478038760 0x7fd47803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:52.478 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.742+0000 7fd494a08700 1 -- 172.21.15.17:0/2699808110 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd48004c9e0 con 0x7fd490105320 2023-07-14T11:52:52.478 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.742+0000 7fd48e7fc700 1 --2- 172.21.15.17:0/2699808110 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fd478038760 0x7fd47803ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:52.478 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.743+0000 7fd48e7fc700 1 --2- 172.21.15.17:0/2699808110 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fd478038760 0x7fd47803ac10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd484006fd0 tx=0x7fd484006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:52.479 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.749+0000 7fd494a08700 1 -- 172.21.15.17:0/2699808110 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd48001a430 con 0x7fd490105320 2023-07-14T11:52:52.479 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.972+0000 7fd495a0a700 1 -- 172.21.15.17:0/2699808110 --> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fd490102ce0 con 0x7fd478038760 2023-07-14T11:52:52.479 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.977+0000 7fd494a08700 1 -- 172.21.15.17:0/2699808110 <== mgr.14120 v2:172.21.15.17:6800/1059284716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fd490102ce0 con 0x7fd478038760 2023-07-14T11:52:52.479 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.980+0000 7fd495a0a700 1 -- 172.21.15.17:0/2699808110 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fd478038760 msgr2=0x7fd47803ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:52.479 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.980+0000 7fd495a0a700 1 --2- 172.21.15.17:0/2699808110 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fd478038760 0x7fd47803ac10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd484006fd0 tx=0x7fd484006e40 comp rx=0 tx=0).stop 2023-07-14T11:52:52.479 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.980+0000 7fd495a0a700 1 -- 172.21.15.17:0/2699808110 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd490105320 msgr2=0x7fd4901990e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:52.479 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.980+0000 7fd495a0a700 1 --2- 172.21.15.17:0/2699808110 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd490105320 0x7fd4901990e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd480003f00 tx=0x7fd480011a30 comp rx=0 tx=0).stop 2023-07-14T11:52:52.480 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.980+0000 7fd495a0a700 1 -- 172.21.15.17:0/2699808110 shutdown_connections 2023-07-14T11:52:52.480 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.980+0000 7fd495a0a700 1 --2- 172.21.15.17:0/2699808110 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fd478038760 0x7fd47803ac10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:52.480 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.980+0000 7fd495a0a700 1 --2- 172.21.15.17:0/2699808110 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd490105320 0x7fd4901990e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:52.480 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.980+0000 7fd495a0a700 1 -- 172.21.15.17:0/2699808110 >> 172.21.15.17:0/2699808110 conn(0x7fd490100970 msgr2=0x7fd4901020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:52.480 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.980+0000 7fd495a0a700 1 -- 172.21.15.17:0/2699808110 shutdown_connections 2023-07-14T11:52:52.480 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:51.980+0000 7fd495a0a700 1 -- 172.21.15.17:0/2699808110 wait complete. 2023-07-14T11:52:52.841 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:52 smithi017 ceph-mon[126339]: from='mgr.14120 172.21.15.17:0/3183437250' entity='mgr.x' 2023-07-14T11:52:52.841 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:52 smithi017 ceph-mon[126339]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:52:52.842 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:52 smithi017 ceph-mon[126339]: Saving service mgr spec with placement count:2 2023-07-14T11:52:52.842 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:52 smithi017 ceph-mon[126339]: from='mgr.14120 172.21.15.17:0/3183437250' entity='mgr.x' 2023-07-14T11:52:52.842 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:52 smithi017 ceph-mon[126339]: from='mgr.14120 172.21.15.17:0/3183437250' entity='mgr.x' 2023-07-14T11:52:53.852 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.144+0000 7fb7a32d1700 1 Processor -- start 2023-07-14T11:52:53.853 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.144+0000 7fb7a32d1700 1 -- start start 2023-07-14T11:52:53.853 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.144+0000 7fb7a32d1700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb79c07be00 0x7fb79c07a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:53.853 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.144+0000 7fb7a32d1700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79c07a840 con 0x7fb79c07be00 2023-07-14T11:52:53.853 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.145+0000 7fb7a106d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb79c07be00 0x7fb79c07a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:53.853 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.145+0000 7fb7a106d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb79c07be00 0x7fb79c07a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46342/0 (socket says 172.21.15.17:46342) 2023-07-14T11:52:53.853 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.145+0000 7fb7a106d700 1 -- 172.21.15.17:0/3874095931 learned_addr learned my addr 172.21.15.17:0/3874095931 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:52:53.853 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.145+0000 7fb7a106d700 1 -- 172.21.15.17:0/3874095931 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb79c07a980 con 0x7fb79c07be00 2023-07-14T11:52:53.854 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.146+0000 7fb7a106d700 1 --2- 172.21.15.17:0/3874095931 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb79c07be00 0x7fb79c07a300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb7980098a0 tx=0x7fb798009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1e949b1ce282723 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:53.854 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.146+0000 7fb793fff700 1 -- 172.21.15.17:0/3874095931 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb798004160 con 0x7fb79c07be00 2023-07-14T11:52:53.854 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.146+0000 7fb793fff700 1 -- 172.21.15.17:0/3874095931 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7980042c0 con 0x7fb79c07be00 2023-07-14T11:52:53.854 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.146+0000 7fb793fff700 1 -- 172.21.15.17:0/3874095931 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb7980119b0 con 0x7fb79c07be00 2023-07-14T11:52:53.854 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.146+0000 7fb7a32d1700 1 -- 172.21.15.17:0/3874095931 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb79c07be00 msgr2=0x7fb79c07a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:53.857 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.146+0000 7fb7a32d1700 1 --2- 172.21.15.17:0/3874095931 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb79c07be00 0x7fb79c07a300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb7980098a0 tx=0x7fb798009bb0 comp rx=0 tx=0).stop 2023-07-14T11:52:53.857 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.147+0000 7fb7a32d1700 1 -- 172.21.15.17:0/3874095931 shutdown_connections 2023-07-14T11:52:53.857 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.147+0000 7fb7a32d1700 1 --2- 172.21.15.17:0/3874095931 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb79c07be00 0x7fb79c07a300 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:53.857 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.147+0000 7fb7a32d1700 1 -- 172.21.15.17:0/3874095931 >> 172.21.15.17:0/3874095931 conn(0x7fb79c100970 msgr2=0x7fb79c102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:53.858 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.147+0000 7fb7a32d1700 1 -- 172.21.15.17:0/3874095931 shutdown_connections 2023-07-14T11:52:53.858 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.147+0000 7fb7a32d1700 1 -- 172.21.15.17:0/3874095931 wait complete. 2023-07-14T11:52:53.858 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.148+0000 7fb7a32d1700 1 Processor -- start 2023-07-14T11:52:53.858 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.148+0000 7fb7a32d1700 1 -- start start 2023-07-14T11:52:53.858 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.149+0000 7fb7a32d1700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb79c07be00 0x7fb79c19b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:53.858 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.149+0000 7fb7a32d1700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79c19b880 con 0x7fb79c07be00 2023-07-14T11:52:53.858 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.149+0000 7fb7a106d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb79c07be00 0x7fb79c19b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:53.858 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.149+0000 7fb7a106d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb79c07be00 0x7fb79c19b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46356/0 (socket says 172.21.15.17:46356) 2023-07-14T11:52:53.859 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.149+0000 7fb7a106d700 1 -- 172.21.15.17:0/242407186 learned_addr learned my addr 172.21.15.17:0/242407186 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:52:53.859 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.150+0000 7fb7a106d700 1 -- 172.21.15.17:0/242407186 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb798009550 con 0x7fb79c07be00 2023-07-14T11:52:53.859 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.150+0000 7fb7a106d700 1 --2- 172.21.15.17:0/242407186 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb79c07be00 0x7fb79c19b340 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb798003e60 tx=0x7fb798011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:53.859 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.150+0000 7fb7927fc700 1 -- 172.21.15.17:0/242407186 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb798003a60 con 0x7fb79c07be00 2023-07-14T11:52:53.859 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.150+0000 7fb7927fc700 1 -- 172.21.15.17:0/242407186 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb798003bc0 con 0x7fb79c07be00 2023-07-14T11:52:53.859 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.150+0000 7fb7927fc700 1 -- 172.21.15.17:0/242407186 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb7980196c0 con 0x7fb79c07be00 2023-07-14T11:52:53.859 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.151+0000 7fb7a32d1700 1 -- 172.21.15.17:0/242407186 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb79c19ba80 con 0x7fb79c07be00 2023-07-14T11:52:53.860 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.151+0000 7fb7a32d1700 1 -- 172.21.15.17:0/242407186 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb79c19bea0 con 0x7fb79c07be00 2023-07-14T11:52:53.860 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.152+0000 7fb7927fc700 1 -- 172.21.15.17:0/242407186 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fb79801ab60 con 0x7fb79c07be00 2023-07-14T11:52:53.860 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.152+0000 7fb7a32d1700 1 -- 172.21.15.17:0/242407186 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb79c061b40 con 0x7fb79c07be00 2023-07-14T11:52:53.860 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.152+0000 7fb7927fc700 1 --2- 172.21.15.17:0/242407186 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fb788038740 0x7fb78803abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:53.860 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.152+0000 7fb7927fc700 1 -- 172.21.15.17:0/242407186 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb79804c9d0 con 0x7fb79c07be00 2023-07-14T11:52:53.860 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.153+0000 7fb7a086c700 1 --2- 172.21.15.17:0/242407186 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fb788038740 0x7fb78803abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:53.860 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.154+0000 7fb7a086c700 1 --2- 172.21.15.17:0/242407186 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fb788038740 0x7fb78803abf0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb78c006fd0 tx=0x7fb78c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:53.860 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.158+0000 7fb7927fc700 1 -- 172.21.15.17:0/242407186 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb798010790 con 0x7fb79c07be00 2023-07-14T11:52:53.861 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.382+0000 7fb7a32d1700 1 -- 172.21.15.17:0/242407186 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7fb79c19e8e0 con 0x7fb79c07be00 2023-07-14T11:52:53.861 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.386+0000 7fb7927fc700 1 -- 172.21.15.17:0/242407186 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7fb79802b350 con 0x7fb79c07be00 2023-07-14T11:52:53.861 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.389+0000 7fb7a32d1700 1 -- 172.21.15.17:0/242407186 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fb788038740 msgr2=0x7fb78803abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:53.861 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.389+0000 7fb7a32d1700 1 --2- 172.21.15.17:0/242407186 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fb788038740 0x7fb78803abf0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb78c006fd0 tx=0x7fb78c006e40 comp rx=0 tx=0).stop 2023-07-14T11:52:53.861 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.389+0000 7fb7a32d1700 1 -- 172.21.15.17:0/242407186 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb79c07be00 msgr2=0x7fb79c19b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:53.861 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.389+0000 7fb7a32d1700 1 --2- 172.21.15.17:0/242407186 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb79c07be00 0x7fb79c19b340 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb798003e60 tx=0x7fb798011a30 comp rx=0 tx=0).stop 2023-07-14T11:52:53.861 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.389+0000 7fb7a32d1700 1 -- 172.21.15.17:0/242407186 shutdown_connections 2023-07-14T11:52:53.862 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.389+0000 7fb7a32d1700 1 --2- 172.21.15.17:0/242407186 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fb788038740 0x7fb78803abf0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:53.862 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.389+0000 7fb7a32d1700 1 --2- 172.21.15.17:0/242407186 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb79c07be00 0x7fb79c19b340 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:53.862 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.389+0000 7fb7a32d1700 1 -- 172.21.15.17:0/242407186 >> 172.21.15.17:0/242407186 conn(0x7fb79c100970 msgr2=0x7fb79c102080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:53.862 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.390+0000 7fb7a32d1700 1 -- 172.21.15.17:0/242407186 shutdown_connections 2023-07-14T11:52:53.862 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:53.390+0000 7fb7a32d1700 1 -- 172.21.15.17:0/242407186 wait complete. 2023-07-14T11:52:54.647 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:54 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/242407186' entity='client.admin' 2023-07-14T11:52:55.334 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.579+0000 7f481bc2e700 1 Processor -- start 2023-07-14T11:52:55.335 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.579+0000 7f481bc2e700 1 -- start start 2023-07-14T11:52:55.335 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.579+0000 7f481bc2e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f481407be40 0x7f481407a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:55.335 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.579+0000 7f481bc2e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481407a880 con 0x7f481407be40 2023-07-14T11:52:55.335 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.580+0000 7f48199ca700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f481407be40 0x7f481407a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:55.335 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.580+0000 7f48199ca700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f481407be40 0x7f481407a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46358/0 (socket says 172.21.15.17:46358) 2023-07-14T11:52:55.335 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.580+0000 7f48199ca700 1 -- 172.21.15.17:0/3775277318 learned_addr learned my addr 172.21.15.17:0/3775277318 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:52:55.335 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.580+0000 7f48199ca700 1 -- 172.21.15.17:0/3775277318 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f481407a9c0 con 0x7f481407be40 2023-07-14T11:52:55.339 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.580+0000 7f48199ca700 1 --2- 172.21.15.17:0/3775277318 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f481407be40 0x7f481407a340 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f48100098a0 tx=0x7f4810009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5fa31a1a1bbcba1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:55.339 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.581+0000 7f48189c8700 1 -- 172.21.15.17:0/3775277318 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4810004160 con 0x7f481407be40 2023-07-14T11:52:55.339 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.581+0000 7f48189c8700 1 -- 172.21.15.17:0/3775277318 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48100042c0 con 0x7f481407be40 2023-07-14T11:52:55.339 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.581+0000 7f48189c8700 1 -- 172.21.15.17:0/3775277318 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f48100119b0 con 0x7f481407be40 2023-07-14T11:52:55.339 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.581+0000 7f481bc2e700 1 -- 172.21.15.17:0/3775277318 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f481407be40 msgr2=0x7f481407a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:55.340 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.581+0000 7f481bc2e700 1 --2- 172.21.15.17:0/3775277318 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f481407be40 0x7f481407a340 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f48100098a0 tx=0x7f4810009bb0 comp rx=0 tx=0).stop 2023-07-14T11:52:55.340 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.581+0000 7f481bc2e700 1 -- 172.21.15.17:0/3775277318 shutdown_connections 2023-07-14T11:52:55.340 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.581+0000 7f481bc2e700 1 --2- 172.21.15.17:0/3775277318 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f481407be40 0x7f481407a340 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:55.340 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.581+0000 7f481bc2e700 1 -- 172.21.15.17:0/3775277318 >> 172.21.15.17:0/3775277318 conn(0x7f4814100970 msgr2=0x7f4814102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:55.340 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.581+0000 7f481bc2e700 1 -- 172.21.15.17:0/3775277318 shutdown_connections 2023-07-14T11:52:55.340 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.581+0000 7f481bc2e700 1 -- 172.21.15.17:0/3775277318 wait complete. 2023-07-14T11:52:55.340 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.582+0000 7f481bc2e700 1 Processor -- start 2023-07-14T11:52:55.340 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.583+0000 7f481bc2e700 1 -- start start 2023-07-14T11:52:55.341 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.583+0000 7f481bc2e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f481407be40 0x7f481419b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:55.341 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.583+0000 7f481bc2e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481419b870 con 0x7f481407be40 2023-07-14T11:52:55.341 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.583+0000 7f48199ca700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f481407be40 0x7f481419b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:55.341 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.583+0000 7f48199ca700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f481407be40 0x7f481419b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46362/0 (socket says 172.21.15.17:46362) 2023-07-14T11:52:55.341 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.583+0000 7f48199ca700 1 -- 172.21.15.17:0/2243292614 learned_addr learned my addr 172.21.15.17:0/2243292614 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:52:55.341 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.584+0000 7f48199ca700 1 -- 172.21.15.17:0/2243292614 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4810009550 con 0x7f481407be40 2023-07-14T11:52:55.341 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.584+0000 7f48199ca700 1 --2- 172.21.15.17:0/2243292614 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f481407be40 0x7f481419b330 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4810006af0 tx=0x7f4810011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:55.342 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.584+0000 7f480affd700 1 -- 172.21.15.17:0/2243292614 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4810003a80 con 0x7f481407be40 2023-07-14T11:52:55.342 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.584+0000 7f480affd700 1 -- 172.21.15.17:0/2243292614 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4810003be0 con 0x7f481407be40 2023-07-14T11:52:55.342 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.584+0000 7f480affd700 1 -- 172.21.15.17:0/2243292614 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f481001a760 con 0x7f481407be40 2023-07-14T11:52:55.342 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.584+0000 7f481bc2e700 1 -- 172.21.15.17:0/2243292614 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f481419ba70 con 0x7f481407be40 2023-07-14T11:52:55.342 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.584+0000 7f481bc2e700 1 -- 172.21.15.17:0/2243292614 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f481419be90 con 0x7f481407be40 2023-07-14T11:52:55.342 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.585+0000 7f480affd700 1 -- 172.21.15.17:0/2243292614 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f4810024b80 con 0x7f481407be40 2023-07-14T11:52:55.342 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.585+0000 7f481bc2e700 1 -- 172.21.15.17:0/2243292614 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4814195310 con 0x7f481407be40 2023-07-14T11:52:55.342 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.586+0000 7f480affd700 1 --2- 172.21.15.17:0/2243292614 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7f4800038710 0x7f480003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:55.343 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.586+0000 7f480affd700 1 -- 172.21.15.17:0/2243292614 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f481004c840 con 0x7f481407be40 2023-07-14T11:52:55.343 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.586+0000 7f48191c9700 1 --2- 172.21.15.17:0/2243292614 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7f4800038710 0x7f480003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:55.343 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.587+0000 7f48191c9700 1 --2- 172.21.15.17:0/2243292614 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7f4800038710 0x7f480003abc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4804006fd0 tx=0x7f4804006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:55.343 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.591+0000 7f480affd700 1 -- 172.21.15.17:0/2243292614 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4810024430 con 0x7f481407be40 2023-07-14T11:52:55.343 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.828+0000 7f481bc2e700 1 -- 172.21.15.17:0/2243292614 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f4814061b40 con 0x7f481407be40 2023-07-14T11:52:55.343 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.831+0000 7f480affd700 1 -- 172.21.15.17:0/2243292614 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f4810018350 con 0x7f481407be40 2023-07-14T11:52:55.343 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.834+0000 7f481bc2e700 1 -- 172.21.15.17:0/2243292614 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7f4800038710 msgr2=0x7f480003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:55.343 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.834+0000 7f481bc2e700 1 --2- 172.21.15.17:0/2243292614 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7f4800038710 0x7f480003abc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4804006fd0 tx=0x7f4804006e40 comp rx=0 tx=0).stop 2023-07-14T11:52:55.344 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.834+0000 7f481bc2e700 1 -- 172.21.15.17:0/2243292614 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f481407be40 msgr2=0x7f481419b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:55.344 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.834+0000 7f481bc2e700 1 --2- 172.21.15.17:0/2243292614 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f481407be40 0x7f481419b330 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4810006af0 tx=0x7f4810011aa0 comp rx=0 tx=0).stop 2023-07-14T11:52:55.344 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.834+0000 7f481bc2e700 1 -- 172.21.15.17:0/2243292614 shutdown_connections 2023-07-14T11:52:55.344 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.834+0000 7f481bc2e700 1 --2- 172.21.15.17:0/2243292614 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7f4800038710 0x7f480003abc0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:55.344 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.834+0000 7f481bc2e700 1 --2- 172.21.15.17:0/2243292614 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f481407be40 0x7f481419b330 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:55.344 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.834+0000 7f481bc2e700 1 -- 172.21.15.17:0/2243292614 >> 172.21.15.17:0/2243292614 conn(0x7f4814100970 msgr2=0x7f4814102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:55.344 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.834+0000 7f481bc2e700 1 -- 172.21.15.17:0/2243292614 shutdown_connections 2023-07-14T11:52:55.345 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:54.834+0000 7f481bc2e700 1 -- 172.21.15.17:0/2243292614 wait complete. 2023-07-14T11:52:55.345 INFO:teuthology.orchestra.run.smithi017.stdout:Enabling the dashboard module... 2023-07-14T11:52:55.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:55 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/2243292614' entity='client.admin' 2023-07-14T11:52:55.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:55 smithi017 ceph-mon[126339]: from='mgr.14120 172.21.15.17:0/3183437250' entity='mgr.x' 2023-07-14T11:52:57.130 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:56 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: ignoring --setuser ceph since I am not root 2023-07-14T11:52:57.130 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:56 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: ignoring --setgroup ceph since I am not root 2023-07-14T11:52:57.131 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:57 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:57.146+0000 7f355ed95200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-14T11:52:57.131 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:56 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/1332522392' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-07-14T11:52:57.131 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:56 smithi017 ceph-mon[126339]: from='mgr.14120 172.21.15.17:0/3183437250' entity='mgr.x' 2023-07-14T11:52:57.329 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:55.961+0000 7fd3fd9e2700 1 Processor -- start 2023-07-14T11:52:57.329 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:55.961+0000 7fd3fd9e2700 1 -- start start 2023-07-14T11:52:57.330 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:55.961+0000 7fd3fd9e2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3f8105320 0x7fd3f8105730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:57.330 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:55.961+0000 7fd3fd9e2700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f8105c70 con 0x7fd3f8105320 2023-07-14T11:52:57.330 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:55.962+0000 7fd3f6ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3f8105320 0x7fd3f8105730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:57.330 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:55.962+0000 7fd3f6ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3f8105320 0x7fd3f8105730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46378/0 (socket says 172.21.15.17:46378) 2023-07-14T11:52:57.331 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:55.962+0000 7fd3f6ffd700 1 -- 172.21.15.17:0/2741538875 learned_addr learned my addr 172.21.15.17:0/2741538875 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:52:57.331 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:55.962+0000 7fd3f6ffd700 1 -- 172.21.15.17:0/2741538875 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f8105db0 con 0x7fd3f8105320 2023-07-14T11:52:57.331 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:55.963+0000 7fd3f6ffd700 1 --2- 172.21.15.17:0/2741538875 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3f8105320 0x7fd3f8105730 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd3e80098a0 tx=0x7fd3e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83388005d7da243b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:57.332 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:55.963+0000 7fd3f5ffb700 1 -- 172.21.15.17:0/2741538875 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd3e8004160 con 0x7fd3f8105320 2023-07-14T11:52:57.332 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:55.963+0000 7fd3f5ffb700 1 -- 172.21.15.17:0/2741538875 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3e80042c0 con 0x7fd3f8105320 2023-07-14T11:52:57.332 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:55.963+0000 7fd3f5ffb700 1 -- 172.21.15.17:0/2741538875 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd3e80119b0 con 0x7fd3f8105320 2023-07-14T11:52:57.333 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:55.964+0000 7fd3fd9e2700 1 -- 172.21.15.17:0/2741538875 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3f8105320 msgr2=0x7fd3f8105730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:57.333 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:55.964+0000 7fd3fd9e2700 1 --2- 172.21.15.17:0/2741538875 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3f8105320 0x7fd3f8105730 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd3e80098a0 tx=0x7fd3e8009bb0 comp rx=0 tx=0).stop 2023-07-14T11:52:57.333 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:55.964+0000 7fd3fd9e2700 1 -- 172.21.15.17:0/2741538875 shutdown_connections 2023-07-14T11:52:57.333 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:55.964+0000 7fd3fd9e2700 1 --2- 172.21.15.17:0/2741538875 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3f8105320 0x7fd3f8105730 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:57.334 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:55.964+0000 7fd3fd9e2700 1 -- 172.21.15.17:0/2741538875 >> 172.21.15.17:0/2741538875 conn(0x7fd3f8100970 msgr2=0x7fd3f8102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:57.334 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:55.964+0000 7fd3fd9e2700 1 -- 172.21.15.17:0/2741538875 shutdown_connections 2023-07-14T11:52:57.334 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:55.964+0000 7fd3fd9e2700 1 -- 172.21.15.17:0/2741538875 wait complete. 2023-07-14T11:52:57.335 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:55.965+0000 7fd3fd9e2700 1 Processor -- start 2023-07-14T11:52:57.335 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:55.966+0000 7fd3fd9e2700 1 -- start start 2023-07-14T11:52:57.335 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:55.966+0000 7fd3fd9e2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3f8105320 0x7fd3f81990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:57.335 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:55.966+0000 7fd3fd9e2700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f8199620 con 0x7fd3f8105320 2023-07-14T11:52:57.336 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:55.966+0000 7fd3f6ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3f8105320 0x7fd3f81990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:57.336 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:55.966+0000 7fd3f6ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3f8105320 0x7fd3f81990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46390/0 (socket says 172.21.15.17:46390) 2023-07-14T11:52:57.336 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:55.966+0000 7fd3f6ffd700 1 -- 172.21.15.17:0/1332522392 learned_addr learned my addr 172.21.15.17:0/1332522392 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:52:57.337 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:55.967+0000 7fd3f6ffd700 1 -- 172.21.15.17:0/1332522392 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3e8009550 con 0x7fd3f8105320 2023-07-14T11:52:57.337 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:55.967+0000 7fd3f6ffd700 1 --2- 172.21.15.17:0/1332522392 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3f8105320 0x7fd3f81990e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8006af0 tx=0x7fd3e8011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:57.337 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:55.967+0000 7fd3fc9e0700 1 -- 172.21.15.17:0/1332522392 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd3e8003a40 con 0x7fd3f8105320 2023-07-14T11:52:57.337 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:55.967+0000 7fd3fd9e2700 1 -- 172.21.15.17:0/1332522392 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3f8199820 con 0x7fd3f8105320 2023-07-14T11:52:57.338 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:55.967+0000 7fd3fc9e0700 1 -- 172.21.15.17:0/1332522392 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3e8003ba0 con 0x7fd3f8105320 2023-07-14T11:52:57.338 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:55.967+0000 7fd3fc9e0700 1 -- 172.21.15.17:0/1332522392 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd3e80196c0 con 0x7fd3f8105320 2023-07-14T11:52:57.338 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:55.968+0000 7fd3fd9e2700 1 -- 172.21.15.17:0/1332522392 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3f8199c40 con 0x7fd3f8105320 2023-07-14T11:52:57.339 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:55.969+0000 7fd3fd9e2700 1 -- 172.21.15.17:0/1332522392 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3f81097b0 con 0x7fd3f8105320 2023-07-14T11:52:57.339 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:55.969+0000 7fd3fc9e0700 1 -- 172.21.15.17:0/1332522392 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fd3e801a430 con 0x7fd3f8105320 2023-07-14T11:52:57.339 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:55.969+0000 7fd3fc9e0700 1 --2- 172.21.15.17:0/1332522392 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fd3e0038710 0x7fd3e003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:57.339 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:55.969+0000 7fd3fc9e0700 1 -- 172.21.15.17:0/1332522392 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd3e804c6e0 con 0x7fd3f8105320 2023-07-14T11:52:57.340 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:55.969+0000 7fd3f67fc700 1 --2- 172.21.15.17:0/1332522392 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fd3e0038710 0x7fd3e003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:57.340 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:55.970+0000 7fd3f67fc700 1 --2- 172.21.15.17:0/1332522392 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fd3e0038710 0x7fd3e003abc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec006fd0 tx=0x7fd3ec006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:57.340 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:55.974+0000 7fd3fc9e0700 1 -- 172.21.15.17:0/1332522392 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd3e8023390 con 0x7fd3f8105320 2023-07-14T11:52:57.340 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:56.256+0000 7fd3fd9e2700 1 -- 172.21.15.17:0/1332522392 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fd3f8061b40 con 0x7fd3f8105320 2023-07-14T11:52:57.341 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:56.835+0000 7fd3fc9e0700 1 -- 172.21.15.17:0/1332522392 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7fd3e801aa80 con 0x7fd3f8105320 2023-07-14T11:52:57.341 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:56.837+0000 7fd3fc9e0700 1 -- 172.21.15.17:0/1332522392 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7fd3e804c440 con 0x7fd3f8105320 2023-07-14T11:52:57.341 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:56.839+0000 7fd3fd9e2700 1 -- 172.21.15.17:0/1332522392 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fd3e0038710 msgr2=0x7fd3e003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:57.341 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:56.839+0000 7fd3fd9e2700 1 --2- 172.21.15.17:0/1332522392 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fd3e0038710 0x7fd3e003abc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec006fd0 tx=0x7fd3ec006e40 comp rx=0 tx=0).stop 2023-07-14T11:52:57.341 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:56.839+0000 7fd3fd9e2700 1 -- 172.21.15.17:0/1332522392 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3f8105320 msgr2=0x7fd3f81990e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:57.341 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:56.839+0000 7fd3fd9e2700 1 --2- 172.21.15.17:0/1332522392 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3f8105320 0x7fd3f81990e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8006af0 tx=0x7fd3e8011a10 comp rx=0 tx=0).stop 2023-07-14T11:52:57.342 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:56.840+0000 7fd3fd9e2700 1 -- 172.21.15.17:0/1332522392 shutdown_connections 2023-07-14T11:52:57.342 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:56.840+0000 7fd3fd9e2700 1 --2- 172.21.15.17:0/1332522392 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fd3e0038710 0x7fd3e003abc0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:57.342 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:56.840+0000 7fd3fd9e2700 1 --2- 172.21.15.17:0/1332522392 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3f8105320 0x7fd3f81990e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:57.342 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:56.840+0000 7fd3fd9e2700 1 -- 172.21.15.17:0/1332522392 >> 172.21.15.17:0/1332522392 conn(0x7fd3f8100970 msgr2=0x7fd3f81020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:57.342 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:56.840+0000 7fd3fd9e2700 1 -- 172.21.15.17:0/1332522392 shutdown_connections 2023-07-14T11:52:57.342 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:56.840+0000 7fd3fd9e2700 1 -- 172.21.15.17:0/1332522392 wait complete. 2023-07-14T11:52:57.397 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:57 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:57.253+0000 7f355ed95200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-14T11:52:57.397 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:57 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:57.385+0000 7f355ed95200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-14T11:52:57.762 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:57 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:57.523+0000 7f355ed95200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-14T11:52:58.050 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:57 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/1332522392' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-07-14T11:52:58.050 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:57 smithi017 ceph-mon[126339]: mgrmap e9: x(active, since 18s) 2023-07-14T11:52:58.050 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:57 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:57.778+0000 7f355ed95200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-14T11:52:58.335 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:58 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:58.219+0000 7f355ed95200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-14T11:52:58.336 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:58 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:58.351+0000 7f355ed95200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-14T11:52:58.635 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:58 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:58.524+0000 7f355ed95200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-14T11:52:58.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:52:58 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/1311939372' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-07-14T11:52:58.897 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:58 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:58.651+0000 7f355ed95200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-14T11:52:58.925 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout { 2023-07-14T11:52:58.925 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-07-14T11:52:58.925 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "available": true, 2023-07-14T11:52:58.925 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-07-14T11:52:58.930 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-07-14T11:52:58.930 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout } 2023-07-14T11:52:58.930 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:57.947+0000 7f0624756700 1 Processor -- start 2023-07-14T11:52:58.930 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:57.947+0000 7f0624756700 1 -- start start 2023-07-14T11:52:58.930 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:57.947+0000 7f0624756700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f061c07be00 0x7f061c07a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:58.931 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:57.947+0000 7f0624756700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061c07a840 con 0x7f061c07be00 2023-07-14T11:52:58.931 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:57.948+0000 7f06224f2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f061c07be00 0x7f061c07a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:58.931 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:57.948+0000 7f06224f2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f061c07be00 0x7f061c07a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46418/0 (socket says 172.21.15.17:46418) 2023-07-14T11:52:58.931 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:57.948+0000 7f06224f2700 1 -- 172.21.15.17:0/1842420530 learned_addr learned my addr 172.21.15.17:0/1842420530 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:52:58.931 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:57.948+0000 7f06224f2700 1 -- 172.21.15.17:0/1842420530 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f061c07a980 con 0x7f061c07be00 2023-07-14T11:52:58.931 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:57.948+0000 7f06224f2700 1 --2- 172.21.15.17:0/1842420530 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f061c07be00 0x7f061c07a300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f06180098a0 tx=0x7f0618009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dce3385b9f88d1a9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:58.931 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:57.948+0000 7f06214f0700 1 -- 172.21.15.17:0/1842420530 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0618004160 con 0x7f061c07be00 2023-07-14T11:52:58.932 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:57.949+0000 7f06214f0700 1 -- 172.21.15.17:0/1842420530 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06180042c0 con 0x7f061c07be00 2023-07-14T11:52:58.932 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:57.949+0000 7f06214f0700 1 -- 172.21.15.17:0/1842420530 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f06180119b0 con 0x7f061c07be00 2023-07-14T11:52:58.932 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:57.949+0000 7f0624756700 1 -- 172.21.15.17:0/1842420530 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f061c07be00 msgr2=0x7f061c07a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:58.932 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:57.949+0000 7f0624756700 1 --2- 172.21.15.17:0/1842420530 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f061c07be00 0x7f061c07a300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f06180098a0 tx=0x7f0618009bb0 comp rx=0 tx=0).stop 2023-07-14T11:52:58.933 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:57.949+0000 7f0624756700 1 -- 172.21.15.17:0/1842420530 shutdown_connections 2023-07-14T11:52:58.933 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:57.949+0000 7f0624756700 1 --2- 172.21.15.17:0/1842420530 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f061c07be00 0x7f061c07a300 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:58.933 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:57.949+0000 7f0624756700 1 -- 172.21.15.17:0/1842420530 >> 172.21.15.17:0/1842420530 conn(0x7f061c100970 msgr2=0x7f061c102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:58.933 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:57.949+0000 7f0624756700 1 -- 172.21.15.17:0/1842420530 shutdown_connections 2023-07-14T11:52:58.933 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:57.949+0000 7f0624756700 1 -- 172.21.15.17:0/1842420530 wait complete. 2023-07-14T11:52:58.933 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:57.950+0000 7f0624756700 1 Processor -- start 2023-07-14T11:52:58.933 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:57.951+0000 7f0624756700 1 -- start start 2023-07-14T11:52:58.934 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:57.951+0000 7f0624756700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f061c07be00 0x7f061c19b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:58.934 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:57.951+0000 7f0624756700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061c19b880 con 0x7f061c07be00 2023-07-14T11:52:58.934 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:57.951+0000 7f06224f2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f061c07be00 0x7f061c19b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:52:58.934 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:57.951+0000 7f06224f2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f061c07be00 0x7f061c19b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46428/0 (socket says 172.21.15.17:46428) 2023-07-14T11:52:58.934 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:57.951+0000 7f06224f2700 1 -- 172.21.15.17:0/1311939372 learned_addr learned my addr 172.21.15.17:0/1311939372 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:52:58.934 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:57.952+0000 7f06224f2700 1 -- 172.21.15.17:0/1311939372 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0618009550 con 0x7f061c07be00 2023-07-14T11:52:58.934 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:57.952+0000 7f06224f2700 1 --2- 172.21.15.17:0/1311939372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f061c07be00 0x7f061c19b340 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0618003e60 tx=0x7f0618011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:52:58.935 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:57.952+0000 7f06137fe700 1 -- 172.21.15.17:0/1311939372 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0618003a60 con 0x7f061c07be00 2023-07-14T11:52:58.935 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:57.952+0000 7f06137fe700 1 -- 172.21.15.17:0/1311939372 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0618003bc0 con 0x7f061c07be00 2023-07-14T11:52:58.935 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:57.952+0000 7f0624756700 1 -- 172.21.15.17:0/1311939372 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f061c19ba80 con 0x7f061c07be00 2023-07-14T11:52:58.935 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:57.952+0000 7f06137fe700 1 -- 172.21.15.17:0/1311939372 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f06180196c0 con 0x7f061c07be00 2023-07-14T11:52:58.935 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:57.952+0000 7f0624756700 1 -- 172.21.15.17:0/1311939372 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f061c19bea0 con 0x7f061c07be00 2023-07-14T11:52:58.935 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:57.954+0000 7f06137fe700 1 -- 172.21.15.17:0/1311939372 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f061801a430 con 0x7f061c07be00 2023-07-14T11:52:58.935 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:57.954+0000 7f0624756700 1 -- 172.21.15.17:0/1311939372 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f061c061b40 con 0x7f061c07be00 2023-07-14T11:52:58.936 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:57.954+0000 7f06137fe700 1 --2- 172.21.15.17:0/1311939372 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7f0608038740 0x7f060803abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:52:58.936 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:57.954+0000 7f06137fe700 1 -- 172.21.15.17:0/1311939372 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f061804c980 con 0x7f061c07be00 2023-07-14T11:52:58.936 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:57.954+0000 7f0621cf1700 1 -- 172.21.15.17:0/1311939372 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7f0608038740 msgr2=0x7f060803abf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/1059284716 2023-07-14T11:52:58.936 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:57.954+0000 7f0621cf1700 1 --2- 172.21.15.17:0/1311939372 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7f0608038740 0x7f060803abf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-14T11:52:58.936 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:57.959+0000 7f06137fe700 1 -- 172.21.15.17:0/1311939372 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0618010560 con 0x7f061c07be00 2023-07-14T11:52:58.936 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:58.155+0000 7f0621cf1700 1 -- 172.21.15.17:0/1311939372 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7f0608038740 msgr2=0x7f060803abf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/1059284716 2023-07-14T11:52:58.936 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:58.155+0000 7f0621cf1700 1 --2- 172.21.15.17:0/1311939372 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7f0608038740 0x7f060803abf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-14T11:52:58.937 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:58.251+0000 7f0624756700 1 -- 172.21.15.17:0/1311939372 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f061c19e9d0 con 0x7f061c07be00 2023-07-14T11:52:58.937 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:58.252+0000 7f06137fe700 1 -- 172.21.15.17:0/1311939372 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f0618010380 con 0x7f061c07be00 2023-07-14T11:52:58.937 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:58.255+0000 7f0624756700 1 -- 172.21.15.17:0/1311939372 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7f0608038740 msgr2=0x7f060803abf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T11:52:58.937 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:58.255+0000 7f0624756700 1 --2- 172.21.15.17:0/1311939372 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7f0608038740 0x7f060803abf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:58.937 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:58.255+0000 7f0624756700 1 -- 172.21.15.17:0/1311939372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f061c07be00 msgr2=0x7f061c19b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:52:58.937 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:58.255+0000 7f0624756700 1 --2- 172.21.15.17:0/1311939372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f061c07be00 0x7f061c19b340 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0618003e60 tx=0x7f0618011a30 comp rx=0 tx=0).stop 2023-07-14T11:52:58.937 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:58.255+0000 7f0624756700 1 -- 172.21.15.17:0/1311939372 shutdown_connections 2023-07-14T11:52:58.938 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:58.255+0000 7f0624756700 1 --2- 172.21.15.17:0/1311939372 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7f0608038740 0x7f060803abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:58.938 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:58.255+0000 7f0624756700 1 --2- 172.21.15.17:0/1311939372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f061c07be00 0x7f061c19b340 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:52:58.938 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:58.255+0000 7f0624756700 1 -- 172.21.15.17:0/1311939372 >> 172.21.15.17:0/1311939372 conn(0x7f061c100970 msgr2=0x7f061c1020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:52:58.938 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:58.256+0000 7f0624756700 1 -- 172.21.15.17:0/1311939372 shutdown_connections 2023-07-14T11:52:58.938 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:58.256+0000 7f0624756700 1 -- 172.21.15.17:0/1311939372 wait complete. 2023-07-14T11:52:58.938 INFO:teuthology.orchestra.run.smithi017.stdout:Waiting for the mgr to restart... 2023-07-14T11:52:58.938 INFO:teuthology.orchestra.run.smithi017.stdout:Waiting for mgr epoch 9... 2023-07-14T11:52:59.647 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:52:59 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:52:59.225+0000 7f355ed95200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-14T11:53:00.397 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:53:00 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:53:00.003+0000 7f355ed95200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-14T11:53:00.397 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:53:00 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:53:00.216+0000 7f355ed95200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-14T11:53:01.521 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:53:01 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:53:01.187+0000 7f355ed95200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-14T11:53:01.521 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:53:01 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:53:01.311+0000 7f355ed95200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-14T11:53:01.522 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:53:01 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:53:01.537+0000 7f355ed95200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-14T11:53:02.147 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:53:01 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:53:01.907+0000 7f355ed95200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-14T11:53:02.647 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:53:02 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:53:02.187+0000 7f355ed95200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-14T11:53:02.647 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:53:02 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:53:02.384+0000 7f355ed95200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-14T11:53:03.147 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:53:02 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:53:02.735+0000 7f355ed95200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-14T11:53:04.098 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:53:03 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:53:03.684+0000 7f355ed95200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-14T11:53:04.098 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:53:03 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:53:03.862+0000 7f355ed95200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-14T11:53:04.397 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:53:04 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:53:04.114+0000 7f355ed95200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-14T11:53:04.397 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:53:04 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:53:04.265+0000 7f355ed95200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-14T11:53:04.897 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:53:04 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:53:04.549+0000 7f355ed95200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-14T11:53:05.897 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 11:53:05 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T11:53:05.636+0000 7f355ed95200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-14T11:53:05.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:05 smithi017 ceph-mon[126339]: Active manager daemon x restarted 2023-07-14T11:53:05.898 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:05 smithi017 ceph-mon[126339]: Activating manager daemon x 2023-07-14T11:53:05.898 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:05 smithi017 ceph-mon[126339]: osdmap e3: 0 total, 0 up, 0 in 2023-07-14T11:53:05.898 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:05 smithi017 ceph-mon[126339]: mgrmap e10: x(active, starting, since 0.00377193s) 2023-07-14T11:53:05.898 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:05 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T11:53:05.898 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:05 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-14T11:53:05.898 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:05 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-14T11:53:05.898 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:05 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-14T11:53:05.898 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:05 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-14T11:53:05.899 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:05 smithi017 ceph-mon[126339]: Manager daemon x is now available 2023-07-14T11:53:05.899 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:05 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:53:05.899 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:05 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:53:07.133 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout { 2023-07-14T11:53:07.133 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-07-14T11:53:07.133 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout "initialized": true 2023-07-14T11:53:07.133 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout } 2023-07-14T11:53:07.133 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:59.527+0000 7fcdf37b9700 1 Processor -- start 2023-07-14T11:53:07.133 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:59.527+0000 7fcdf37b9700 1 -- start start 2023-07-14T11:53:07.134 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:59.527+0000 7fcdf37b9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdec07be00 0x7fcdec07a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:07.134 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:59.527+0000 7fcdf37b9700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdec07a840 con 0x7fcdec07be00 2023-07-14T11:53:07.134 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:59.528+0000 7fcdf1555700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdec07be00 0x7fcdec07a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:07.134 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:59.528+0000 7fcdf1555700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdec07be00 0x7fcdec07a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46430/0 (socket says 172.21.15.17:46430) 2023-07-14T11:53:07.134 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:59.528+0000 7fcdf1555700 1 -- 172.21.15.17:0/2395471842 learned_addr learned my addr 172.21.15.17:0/2395471842 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:53:07.134 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:59.528+0000 7fcdf1555700 1 -- 172.21.15.17:0/2395471842 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdec07a980 con 0x7fcdec07be00 2023-07-14T11:53:07.134 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:59.528+0000 7fcdf1555700 1 --2- 172.21.15.17:0/2395471842 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdec07be00 0x7fcdec07a300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcde80098a0 tx=0x7fcde8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc00c2875bd9288 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:07.135 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:59.529+0000 7fcde3fff700 1 -- 172.21.15.17:0/2395471842 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcde8004160 con 0x7fcdec07be00 2023-07-14T11:53:07.135 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:59.529+0000 7fcde3fff700 1 -- 172.21.15.17:0/2395471842 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcde80042c0 con 0x7fcdec07be00 2023-07-14T11:53:07.135 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:59.529+0000 7fcde3fff700 1 -- 172.21.15.17:0/2395471842 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcde80119b0 con 0x7fcdec07be00 2023-07-14T11:53:07.135 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:59.529+0000 7fcdf37b9700 1 -- 172.21.15.17:0/2395471842 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdec07be00 msgr2=0x7fcdec07a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:07.135 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:59.529+0000 7fcdf37b9700 1 --2- 172.21.15.17:0/2395471842 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdec07be00 0x7fcdec07a300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcde80098a0 tx=0x7fcde8009bb0 comp rx=0 tx=0).stop 2023-07-14T11:53:07.135 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:59.529+0000 7fcdf37b9700 1 -- 172.21.15.17:0/2395471842 shutdown_connections 2023-07-14T11:53:07.135 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:59.529+0000 7fcdf37b9700 1 --2- 172.21.15.17:0/2395471842 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdec07be00 0x7fcdec07a300 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:07.135 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:59.529+0000 7fcdf37b9700 1 -- 172.21.15.17:0/2395471842 >> 172.21.15.17:0/2395471842 conn(0x7fcdec100970 msgr2=0x7fcdec102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:07.136 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:59.530+0000 7fcdf37b9700 1 -- 172.21.15.17:0/2395471842 shutdown_connections 2023-07-14T11:53:07.136 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:59.530+0000 7fcdf37b9700 1 -- 172.21.15.17:0/2395471842 wait complete. 2023-07-14T11:53:07.136 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:59.531+0000 7fcdf37b9700 1 Processor -- start 2023-07-14T11:53:07.136 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:59.531+0000 7fcdf37b9700 1 -- start start 2023-07-14T11:53:07.136 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:59.532+0000 7fcdf37b9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdec07be00 0x7fcdec19b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:07.136 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:59.532+0000 7fcdf37b9700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdec19b880 con 0x7fcdec07be00 2023-07-14T11:53:07.136 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:59.532+0000 7fcdf1555700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdec07be00 0x7fcdec19b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:07.137 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:59.532+0000 7fcdf1555700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdec07be00 0x7fcdec19b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46440/0 (socket says 172.21.15.17:46440) 2023-07-14T11:53:07.137 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:59.532+0000 7fcdf1555700 1 -- 172.21.15.17:0/3161802646 learned_addr learned my addr 172.21.15.17:0/3161802646 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:53:07.137 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:59.532+0000 7fcdf1555700 1 -- 172.21.15.17:0/3161802646 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde8009550 con 0x7fcdec07be00 2023-07-14T11:53:07.137 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:59.533+0000 7fcdf1555700 1 --2- 172.21.15.17:0/3161802646 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdec07be00 0x7fcdec19b340 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcde8003e60 tx=0x7fcde8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:07.137 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:59.533+0000 7fcde27fc700 1 -- 172.21.15.17:0/3161802646 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcde8003a60 con 0x7fcdec07be00 2023-07-14T11:53:07.137 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:59.533+0000 7fcde27fc700 1 -- 172.21.15.17:0/3161802646 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcde8003bc0 con 0x7fcdec07be00 2023-07-14T11:53:07.137 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:59.533+0000 7fcdf37b9700 1 -- 172.21.15.17:0/3161802646 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdec19ba80 con 0x7fcdec07be00 2023-07-14T11:53:07.137 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:59.533+0000 7fcde27fc700 1 -- 172.21.15.17:0/3161802646 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcde80196c0 con 0x7fcdec07be00 2023-07-14T11:53:07.138 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:59.533+0000 7fcdf37b9700 1 -- 172.21.15.17:0/3161802646 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdec19bea0 con 0x7fcdec07be00 2023-07-14T11:53:07.138 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:59.534+0000 7fcde27fc700 1 -- 172.21.15.17:0/3161802646 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7fcde801ab60 con 0x7fcdec07be00 2023-07-14T11:53:07.138 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:59.535+0000 7fcde27fc700 1 --2- 172.21.15.17:0/3161802646 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fcdd8038790 0x7fcdd803ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:07.138 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:59.535+0000 7fcde27fc700 1 -- 172.21.15.17:0/3161802646 --> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcdd803b350 con 0x7fcdd8038790 2023-07-14T11:53:07.138 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:59.535+0000 7fcde27fc700 1 -- 172.21.15.17:0/3161802646 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fcde804ca00 con 0x7fcdec07be00 2023-07-14T11:53:07.138 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:59.535+0000 7fcdf0d54700 1 -- 172.21.15.17:0/3161802646 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fcdd8038790 msgr2=0x7fcdd803ac40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/1059284716 2023-07-14T11:53:07.138 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:59.535+0000 7fcdf0d54700 1 --2- 172.21.15.17:0/3161802646 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fcdd8038790 0x7fcdd803ac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-14T11:53:07.139 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:59.736+0000 7fcdf0d54700 1 -- 172.21.15.17:0/3161802646 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fcdd8038790 msgr2=0x7fcdd803ac40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/1059284716 2023-07-14T11:53:07.139 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:52:59.736+0000 7fcdf0d54700 1 --2- 172.21.15.17:0/3161802646 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fcdd8038790 0x7fcdd803ac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-14T11:53:07.139 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:00.137+0000 7fcdf0d54700 1 -- 172.21.15.17:0/3161802646 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fcdd8038790 msgr2=0x7fcdd803ac40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/1059284716 2023-07-14T11:53:07.139 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:00.137+0000 7fcdf0d54700 1 --2- 172.21.15.17:0/3161802646 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fcdd8038790 0x7fcdd803ac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-07-14T11:53:07.139 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:00.938+0000 7fcdf0d54700 1 -- 172.21.15.17:0/3161802646 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fcdd8038790 msgr2=0x7fcdd803ac40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/1059284716 2023-07-14T11:53:07.139 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:00.938+0000 7fcdf0d54700 1 --2- 172.21.15.17:0/3161802646 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fcdd8038790 0x7fcdd803ac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-07-14T11:53:07.139 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:02.539+0000 7fcdf0d54700 1 -- 172.21.15.17:0/3161802646 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fcdd8038790 msgr2=0x7fcdd803ac40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:6800/1059284716 2023-07-14T11:53:07.139 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:02.539+0000 7fcdf0d54700 1 --2- 172.21.15.17:0/3161802646 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fcdd8038790 0x7fcdd803ac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-07-14T11:53:07.140 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:05.641+0000 7fcde27fc700 1 -- 172.21.15.17:0/3161802646 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mgrmap(e 10) v1 ==== 45263+0+0 (secure 0 0 0) 0x7fcde801a7b0 con 0x7fcdec07be00 2023-07-14T11:53:07.140 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:05.641+0000 7fcde27fc700 1 -- 172.21.15.17:0/3161802646 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fcdd8038790 msgr2=0x7fcdd803ac40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T11:53:07.140 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:05.641+0000 7fcde27fc700 1 --2- 172.21.15.17:0/3161802646 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fcdd8038790 0x7fcdd803ac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:07.140 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:06.643+0000 7fcde27fc700 1 -- 172.21.15.17:0/3161802646 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mgrmap(e 11) v1 ==== 45390+0+0 (secure 0 0 0) 0x7fcde804e750 con 0x7fcdec07be00 2023-07-14T11:53:07.140 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:06.643+0000 7fcde27fc700 1 --2- 172.21.15.17:0/3161802646 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fcdd803bff0 0x7fcdd803e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:07.140 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:06.644+0000 7fcde27fc700 1 -- 172.21.15.17:0/3161802646 --> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcdd803b350 con 0x7fcdd803bff0 2023-07-14T11:53:07.141 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:06.644+0000 7fcdf0d54700 1 --2- 172.21.15.17:0/3161802646 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fcdd803bff0 0x7fcdd803e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:07.141 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:06.645+0000 7fcdf0d54700 1 --2- 172.21.15.17:0/3161802646 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fcdd803bff0 0x7fcdd803e3d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fcddc003a10 tx=0x7fcddc009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:07.141 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:06.646+0000 7fcde27fc700 1 -- 172.21.15.17:0/3161802646 <== mgr.14152 v2:172.21.15.17:6800/4237449858 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7fcdd803b350 con 0x7fcdd803bff0 2023-07-14T11:53:07.141 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:06.653+0000 7fcdf37b9700 1 -- 172.21.15.17:0/3161802646 --> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fcdec19c680 con 0x7fcdd803bff0 2023-07-14T11:53:07.141 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:06.653+0000 7fcde27fc700 1 -- 172.21.15.17:0/3161802646 <== mgr.14152 v2:172.21.15.17:6800/4237449858 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fcdec19c680 con 0x7fcdd803bff0 2023-07-14T11:53:07.141 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:06.653+0000 7fcdf37b9700 1 -- 172.21.15.17:0/3161802646 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fcdd803bff0 msgr2=0x7fcdd803e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:07.141 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:06.653+0000 7fcdf37b9700 1 --2- 172.21.15.17:0/3161802646 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fcdd803bff0 0x7fcdd803e3d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fcddc003a10 tx=0x7fcddc009270 comp rx=0 tx=0).stop 2023-07-14T11:53:07.141 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:06.653+0000 7fcdf37b9700 1 -- 172.21.15.17:0/3161802646 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdec07be00 msgr2=0x7fcdec19b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:07.142 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:06.653+0000 7fcdf37b9700 1 --2- 172.21.15.17:0/3161802646 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdec07be00 0x7fcdec19b340 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcde8003e60 tx=0x7fcde8011a30 comp rx=0 tx=0).stop 2023-07-14T11:53:07.142 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:06.654+0000 7fcdf37b9700 1 -- 172.21.15.17:0/3161802646 shutdown_connections 2023-07-14T11:53:07.142 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:06.654+0000 7fcdf37b9700 1 --2- 172.21.15.17:0/3161802646 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fcdd803bff0 0x7fcdd803e3d0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:07.142 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:06.654+0000 7fcdf37b9700 1 --2- 172.21.15.17:0/3161802646 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdec07be00 0x7fcdec19b340 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:07.143 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:06.654+0000 7fcdf37b9700 1 --2- 172.21.15.17:0/3161802646 >> [v2:172.21.15.17:6800/1059284716,v1:172.21.15.17:6801/1059284716] conn(0x7fcdd8038790 0x7fcdd803ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:07.143 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:06.654+0000 7fcdf37b9700 1 -- 172.21.15.17:0/3161802646 >> 172.21.15.17:0/3161802646 conn(0x7fcdec100970 msgr2=0x7fcdec1020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:07.143 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:06.654+0000 7fcdf37b9700 1 -- 172.21.15.17:0/3161802646 shutdown_connections 2023-07-14T11:53:07.144 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:06.654+0000 7fcdf37b9700 1 -- 172.21.15.17:0/3161802646 wait complete. 2023-07-14T11:53:07.144 INFO:teuthology.orchestra.run.smithi017.stdout:mgr epoch 9 is available 2023-07-14T11:53:07.144 INFO:teuthology.orchestra.run.smithi017.stdout:Generating a dashboard self-signed certificate... 2023-07-14T11:53:07.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:06 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-14T11:53:07.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:06 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-14T11:53:07.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:06 smithi017 ceph-mon[126339]: mgrmap e11: x(active, since 1.00724s) 2023-07-14T11:53:07.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:07 smithi017 ceph-mon[126339]: [14/Jul/2023:11:53:06] ENGINE Bus STARTING 2023-07-14T11:53:07.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:07 smithi017 ceph-mon[126339]: [14/Jul/2023:11:53:06] ENGINE Serving on http://172.21.15.17:8765 2023-07-14T11:53:07.898 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:07 smithi017 ceph-mon[126339]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-07-14T11:53:07.898 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:07 smithi017 ceph-mon[126339]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-07-14T11:53:07.898 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:07 smithi017 ceph-mon[126339]: [14/Jul/2023:11:53:06] ENGINE Serving on https://172.21.15.17:7150 2023-07-14T11:53:07.898 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:07 smithi017 ceph-mon[126339]: [14/Jul/2023:11:53:06] ENGINE Bus STARTED 2023-07-14T11:53:08.813 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-07-14T11:53:08.814 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:07.983+0000 7f6a10990700 1 Processor -- start 2023-07-14T11:53:08.814 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:07.984+0000 7f6a10990700 1 -- start start 2023-07-14T11:53:08.814 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:07.984+0000 7f6a10990700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a0c105e90 0x7f6a0c108270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:08.814 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:07.984+0000 7f6a10990700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0c079d40 con 0x7f6a0c105e90 2023-07-14T11:53:08.814 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:07.984+0000 7f6a0a59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a0c105e90 0x7f6a0c108270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:08.814 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:07.985+0000 7f6a0a59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a0c105e90 0x7f6a0c108270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49904/0 (socket says 172.21.15.17:49904) 2023-07-14T11:53:08.818 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:07.985+0000 7f6a0a59c700 1 -- 172.21.15.17:0/972755363 learned_addr learned my addr 172.21.15.17:0/972755363 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:53:08.818 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:07.985+0000 7f6a0a59c700 1 -- 172.21.15.17:0/972755363 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a0c079e80 con 0x7f6a0c105e90 2023-07-14T11:53:08.818 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:07.985+0000 7f6a0a59c700 1 --2- 172.21.15.17:0/972755363 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a0c105e90 0x7f6a0c108270 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f69fc0098a0 tx=0x7f69fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a02ae5463fd04da server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:08.819 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:07.985+0000 7f6a0959a700 1 -- 172.21.15.17:0/972755363 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f69fc004160 con 0x7f6a0c105e90 2023-07-14T11:53:08.819 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:07.985+0000 7f6a0959a700 1 -- 172.21.15.17:0/972755363 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69fc0042c0 con 0x7f6a0c105e90 2023-07-14T11:53:08.819 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:07.985+0000 7f6a0959a700 1 -- 172.21.15.17:0/972755363 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f69fc0119b0 con 0x7f6a0c105e90 2023-07-14T11:53:08.819 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:07.986+0000 7f6a10990700 1 -- 172.21.15.17:0/972755363 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a0c105e90 msgr2=0x7f6a0c108270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:08.819 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:07.986+0000 7f6a10990700 1 --2- 172.21.15.17:0/972755363 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a0c105e90 0x7f6a0c108270 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f69fc0098a0 tx=0x7f69fc009bb0 comp rx=0 tx=0).stop 2023-07-14T11:53:08.820 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:07.986+0000 7f6a10990700 1 -- 172.21.15.17:0/972755363 shutdown_connections 2023-07-14T11:53:08.820 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:07.986+0000 7f6a10990700 1 --2- 172.21.15.17:0/972755363 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a0c105e90 0x7f6a0c108270 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:08.820 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:07.986+0000 7f6a10990700 1 -- 172.21.15.17:0/972755363 >> 172.21.15.17:0/972755363 conn(0x7f6a0c0ff890 msgr2=0x7f6a0c101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:08.820 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:07.986+0000 7f6a10990700 1 -- 172.21.15.17:0/972755363 shutdown_connections 2023-07-14T11:53:08.820 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:07.986+0000 7f6a10990700 1 -- 172.21.15.17:0/972755363 wait complete. 2023-07-14T11:53:08.820 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:07.988+0000 7f6a10990700 1 Processor -- start 2023-07-14T11:53:08.820 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:07.988+0000 7f6a10990700 1 -- start start 2023-07-14T11:53:08.821 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:07.988+0000 7f6a10990700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a0c105e90 0x7f6a0c1057f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:08.821 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:07.988+0000 7f6a10990700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0c105d30 con 0x7f6a0c105e90 2023-07-14T11:53:08.821 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:07.988+0000 7f6a0a59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a0c105e90 0x7f6a0c1057f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:08.821 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:07.989+0000 7f6a0a59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a0c105e90 0x7f6a0c1057f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49914/0 (socket says 172.21.15.17:49914) 2023-07-14T11:53:08.821 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:07.989+0000 7f6a0a59c700 1 -- 172.21.15.17:0/4219033407 learned_addr learned my addr 172.21.15.17:0/4219033407 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:53:08.821 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:07.989+0000 7f6a0a59c700 1 -- 172.21.15.17:0/4219033407 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69fc009550 con 0x7f6a0c105e90 2023-07-14T11:53:08.821 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:07.989+0000 7f6a0a59c700 1 --2- 172.21.15.17:0/4219033407 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a0c105e90 0x7f6a0c1057f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f69fc003e60 tx=0x7f69fc011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:08.822 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:07.989+0000 7f69fb7fe700 1 -- 172.21.15.17:0/4219033407 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f69fc003a60 con 0x7f6a0c105e90 2023-07-14T11:53:08.822 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:07.989+0000 7f69fb7fe700 1 -- 172.21.15.17:0/4219033407 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69fc003bc0 con 0x7f6a0c105e90 2023-07-14T11:53:08.822 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:07.989+0000 7f69fb7fe700 1 -- 172.21.15.17:0/4219033407 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f69fc0196c0 con 0x7f6a0c105e90 2023-07-14T11:53:08.822 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:07.989+0000 7f6a10990700 1 -- 172.21.15.17:0/4219033407 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a0c103e40 con 0x7f6a0c105e90 2023-07-14T11:53:08.822 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:07.989+0000 7f6a10990700 1 -- 172.21.15.17:0/4219033407 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a0c104260 con 0x7f6a0c105e90 2023-07-14T11:53:08.822 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:07.990+0000 7f6a10990700 1 -- 172.21.15.17:0/4219033407 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a0c109460 con 0x7f6a0c105e90 2023-07-14T11:53:08.822 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:07.991+0000 7f69fb7fe700 1 -- 172.21.15.17:0/4219033407 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f69fc01ab60 con 0x7f6a0c105e90 2023-07-14T11:53:08.822 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:07.991+0000 7f69fb7fe700 1 --2- 172.21.15.17:0/4219033407 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f69f4038790 0x7f69f403ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:08.823 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:07.991+0000 7f69fb7fe700 1 -- 172.21.15.17:0/4219033407 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f69fc04cae0 con 0x7f6a0c105e90 2023-07-14T11:53:08.823 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:07.991+0000 7f6a09d9b700 1 --2- 172.21.15.17:0/4219033407 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f69f4038790 0x7f69f403ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:08.823 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:07.992+0000 7f6a09d9b700 1 --2- 172.21.15.17:0/4219033407 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f69f4038790 0x7f69f403ac40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6a00006fd0 tx=0x7f6a00006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:08.823 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:07.997+0000 7f69fb7fe700 1 -- 172.21.15.17:0/4219033407 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f69fc018b00 con 0x7f6a0c105e90 2023-07-14T11:53:08.823 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:08.241+0000 7f6a10990700 1 -- 172.21.15.17:0/4219033407 --> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a0c101be0 con 0x7f69f4038790 2023-07-14T11:53:08.823 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:08.358+0000 7f69fb7fe700 1 -- 172.21.15.17:0/4219033407 <== mgr.14152 v2:172.21.15.17:6800/4237449858 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f6a0c101be0 con 0x7f69f4038790 2023-07-14T11:53:08.823 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:08.361+0000 7f6a10990700 1 -- 172.21.15.17:0/4219033407 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f69f4038790 msgr2=0x7f69f403ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:08.824 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:08.361+0000 7f6a10990700 1 --2- 172.21.15.17:0/4219033407 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f69f4038790 0x7f69f403ac40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6a00006fd0 tx=0x7f6a00006e40 comp rx=0 tx=0).stop 2023-07-14T11:53:08.824 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:08.361+0000 7f6a10990700 1 -- 172.21.15.17:0/4219033407 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a0c105e90 msgr2=0x7f6a0c1057f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:08.824 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:08.361+0000 7f6a10990700 1 --2- 172.21.15.17:0/4219033407 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a0c105e90 0x7f6a0c1057f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f69fc003e60 tx=0x7f69fc011a30 comp rx=0 tx=0).stop 2023-07-14T11:53:08.824 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:08.361+0000 7f6a10990700 1 -- 172.21.15.17:0/4219033407 shutdown_connections 2023-07-14T11:53:08.824 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:08.361+0000 7f6a10990700 1 --2- 172.21.15.17:0/4219033407 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f69f4038790 0x7f69f403ac40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:08.824 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:08.361+0000 7f6a10990700 1 --2- 172.21.15.17:0/4219033407 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a0c105e90 0x7f6a0c1057f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:08.824 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:08.361+0000 7f6a10990700 1 -- 172.21.15.17:0/4219033407 >> 172.21.15.17:0/4219033407 conn(0x7f6a0c0ff890 msgr2=0x7f6a0c100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:08.824 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:08.361+0000 7f6a10990700 1 -- 172.21.15.17:0/4219033407 shutdown_connections 2023-07-14T11:53:08.825 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:08.361+0000 7f6a10990700 1 -- 172.21.15.17:0/4219033407 wait complete. 2023-07-14T11:53:08.825 INFO:teuthology.orchestra.run.smithi017.stdout:Creating initial admin user... 2023-07-14T11:53:09.122 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:08 smithi017 ceph-mon[126339]: mgrmap e12: x(active, since 2s) 2023-07-14T11:53:09.123 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:08 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:09.123 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:08 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:09.123 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:08 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:09.946 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:09 smithi017 ceph-mon[126339]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:53:10.404 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$ie2ra40DolzxVGlpSctnO.FY4/ytDD8J16frhIGLVDndN5suEoR2G", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1689335589, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-07-14T11:53:10.404 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.432+0000 7f3ec8ce9700 1 Processor -- start 2023-07-14T11:53:10.405 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.432+0000 7f3ec8ce9700 1 -- start start 2023-07-14T11:53:10.405 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.432+0000 7f3ec8ce9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ec407be00 0x7f3ec407a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:10.405 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.432+0000 7f3ec8ce9700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec407a840 con 0x7f3ec407be00 2023-07-14T11:53:10.405 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.433+0000 7f3ec259c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ec407be00 0x7f3ec407a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:10.405 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.433+0000 7f3ec259c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ec407be00 0x7f3ec407a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49928/0 (socket says 172.21.15.17:49928) 2023-07-14T11:53:10.405 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.433+0000 7f3ec259c700 1 -- 172.21.15.17:0/2284238263 learned_addr learned my addr 172.21.15.17:0/2284238263 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:53:10.405 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.433+0000 7f3ec259c700 1 -- 172.21.15.17:0/2284238263 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec407a980 con 0x7f3ec407be00 2023-07-14T11:53:10.406 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.433+0000 7f3ec259c700 1 --2- 172.21.15.17:0/2284238263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ec407be00 0x7f3ec407a300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3eb40098a0 tx=0x7f3eb4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd4a71ad0fb41822 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:10.406 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.434+0000 7f3ec159a700 1 -- 172.21.15.17:0/2284238263 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3eb4004160 con 0x7f3ec407be00 2023-07-14T11:53:10.406 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.434+0000 7f3ec159a700 1 -- 172.21.15.17:0/2284238263 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3eb40042c0 con 0x7f3ec407be00 2023-07-14T11:53:10.406 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.434+0000 7f3ec159a700 1 -- 172.21.15.17:0/2284238263 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3eb40119b0 con 0x7f3ec407be00 2023-07-14T11:53:10.406 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.434+0000 7f3ec8ce9700 1 -- 172.21.15.17:0/2284238263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ec407be00 msgr2=0x7f3ec407a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:10.406 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.434+0000 7f3ec8ce9700 1 --2- 172.21.15.17:0/2284238263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ec407be00 0x7f3ec407a300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3eb40098a0 tx=0x7f3eb4009bb0 comp rx=0 tx=0).stop 2023-07-14T11:53:10.406 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.434+0000 7f3ec8ce9700 1 -- 172.21.15.17:0/2284238263 shutdown_connections 2023-07-14T11:53:10.407 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.434+0000 7f3ec8ce9700 1 --2- 172.21.15.17:0/2284238263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ec407be00 0x7f3ec407a300 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:10.407 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.434+0000 7f3ec8ce9700 1 -- 172.21.15.17:0/2284238263 >> 172.21.15.17:0/2284238263 conn(0x7f3ec4100970 msgr2=0x7f3ec4102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:10.407 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.434+0000 7f3ec8ce9700 1 -- 172.21.15.17:0/2284238263 shutdown_connections 2023-07-14T11:53:10.407 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.434+0000 7f3ec8ce9700 1 -- 172.21.15.17:0/2284238263 wait complete. 2023-07-14T11:53:10.407 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.436+0000 7f3ec8ce9700 1 Processor -- start 2023-07-14T11:53:10.407 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.436+0000 7f3ec8ce9700 1 -- start start 2023-07-14T11:53:10.407 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.436+0000 7f3ec8ce9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ec407be00 0x7f3ec419b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:10.407 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.436+0000 7f3ec8ce9700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec419b880 con 0x7f3ec407be00 2023-07-14T11:53:10.408 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.437+0000 7f3ec259c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ec407be00 0x7f3ec419b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:10.408 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.437+0000 7f3ec259c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ec407be00 0x7f3ec419b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49934/0 (socket says 172.21.15.17:49934) 2023-07-14T11:53:10.408 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.437+0000 7f3ec259c700 1 -- 172.21.15.17:0/3793862294 learned_addr learned my addr 172.21.15.17:0/3793862294 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:53:10.408 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.437+0000 7f3ec259c700 1 -- 172.21.15.17:0/3793862294 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb4009550 con 0x7f3ec407be00 2023-07-14T11:53:10.408 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.437+0000 7f3ec259c700 1 --2- 172.21.15.17:0/3793862294 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ec407be00 0x7f3ec419b340 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4003e60 tx=0x7f3eb4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:10.408 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.438+0000 7f3eb37fe700 1 -- 172.21.15.17:0/3793862294 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3eb4003a60 con 0x7f3ec407be00 2023-07-14T11:53:10.409 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.438+0000 7f3eb37fe700 1 -- 172.21.15.17:0/3793862294 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3eb4003bc0 con 0x7f3ec407be00 2023-07-14T11:53:10.409 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.438+0000 7f3eb37fe700 1 -- 172.21.15.17:0/3793862294 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3eb40196c0 con 0x7f3ec407be00 2023-07-14T11:53:10.409 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.438+0000 7f3ec8ce9700 1 -- 172.21.15.17:0/3793862294 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ec419ba80 con 0x7f3ec407be00 2023-07-14T11:53:10.409 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.438+0000 7f3ec8ce9700 1 -- 172.21.15.17:0/3793862294 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ec419bea0 con 0x7f3ec407be00 2023-07-14T11:53:10.409 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.439+0000 7f3eb37fe700 1 -- 172.21.15.17:0/3793862294 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f3eb4019980 con 0x7f3ec407be00 2023-07-14T11:53:10.409 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.439+0000 7f3ec8ce9700 1 -- 172.21.15.17:0/3793862294 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ec4061b40 con 0x7f3ec407be00 2023-07-14T11:53:10.409 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.440+0000 7f3eb37fe700 1 --2- 172.21.15.17:0/3793862294 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3eac038740 0x7f3eac03abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:10.410 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.440+0000 7f3eb37fe700 1 -- 172.21.15.17:0/3793862294 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3eb404cd10 con 0x7f3ec407be00 2023-07-14T11:53:10.410 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.440+0000 7f3ec1d9b700 1 --2- 172.21.15.17:0/3793862294 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3eac038740 0x7f3eac03abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:10.410 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.441+0000 7f3ec1d9b700 1 --2- 172.21.15.17:0/3793862294 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3eac038740 0x7f3eac03abf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8006fd0 tx=0x7f3eb8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:10.410 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.445+0000 7f3eb37fe700 1 -- 172.21.15.17:0/3793862294 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3eb4052020 con 0x7f3ec407be00 2023-07-14T11:53:10.410 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.690+0000 7f3ec8ce9700 1 -- 172.21.15.17:0/3793862294 --> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f3ec419e9a0 con 0x7f3eac038740 2023-07-14T11:53:10.410 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.937+0000 7f3eb37fe700 1 -- 172.21.15.17:0/3793862294 <== mgr.14152 v2:172.21.15.17:6800/4237449858 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f3ec419e9a0 con 0x7f3eac038740 2023-07-14T11:53:10.410 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.940+0000 7f3ec8ce9700 1 -- 172.21.15.17:0/3793862294 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3eac038740 msgr2=0x7f3eac03abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:10.411 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.940+0000 7f3ec8ce9700 1 --2- 172.21.15.17:0/3793862294 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3eac038740 0x7f3eac03abf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8006fd0 tx=0x7f3eb8006e40 comp rx=0 tx=0).stop 2023-07-14T11:53:10.411 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.940+0000 7f3ec8ce9700 1 -- 172.21.15.17:0/3793862294 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ec407be00 msgr2=0x7f3ec419b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:10.411 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.940+0000 7f3ec8ce9700 1 --2- 172.21.15.17:0/3793862294 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ec407be00 0x7f3ec419b340 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4003e60 tx=0x7f3eb4011a30 comp rx=0 tx=0).stop 2023-07-14T11:53:10.411 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.940+0000 7f3ec8ce9700 1 -- 172.21.15.17:0/3793862294 shutdown_connections 2023-07-14T11:53:10.411 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.940+0000 7f3ec8ce9700 1 --2- 172.21.15.17:0/3793862294 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3eac038740 0x7f3eac03abf0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:10.411 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.940+0000 7f3ec8ce9700 1 --2- 172.21.15.17:0/3793862294 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ec407be00 0x7f3ec419b340 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:10.411 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.940+0000 7f3ec8ce9700 1 -- 172.21.15.17:0/3793862294 >> 172.21.15.17:0/3793862294 conn(0x7f3ec4100970 msgr2=0x7f3ec4102080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:10.411 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.940+0000 7f3ec8ce9700 1 -- 172.21.15.17:0/3793862294 shutdown_connections 2023-07-14T11:53:10.412 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:09.940+0000 7f3ec8ce9700 1 -- 172.21.15.17:0/3793862294 wait complete. 2023-07-14T11:53:10.412 INFO:teuthology.orchestra.run.smithi017.stdout:Fetching dashboard port number... 2023-07-14T11:53:11.112 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:10 smithi017 ceph-mon[126339]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:53:11.113 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:10 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:11.113 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:10 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:12.086 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stdout 8443 2023-07-14T11:53:12.087 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.053+0000 7f38ae435700 1 Processor -- start 2023-07-14T11:53:12.087 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.053+0000 7f38ae435700 1 -- start start 2023-07-14T11:53:12.087 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.053+0000 7f38ae435700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38a807bde0 0x7f38a807a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:12.087 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.054+0000 7f38ae435700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a807a820 con 0x7f38a807bde0 2023-07-14T11:53:12.087 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.054+0000 7f38a7fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38a807bde0 0x7f38a807a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:12.088 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.054+0000 7f38a7fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38a807bde0 0x7f38a807a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49946/0 (socket says 172.21.15.17:49946) 2023-07-14T11:53:12.088 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.054+0000 7f38a7fff700 1 -- 172.21.15.17:0/2450787933 learned_addr learned my addr 172.21.15.17:0/2450787933 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:53:12.088 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.054+0000 7f38a7fff700 1 -- 172.21.15.17:0/2450787933 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a807a960 con 0x7f38a807bde0 2023-07-14T11:53:12.088 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.054+0000 7f38a7fff700 1 --2- 172.21.15.17:0/2450787933 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38a807bde0 0x7f38a807a2e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f38980098a0 tx=0x7f3898009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77bcbc0f30cc0e0c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:12.088 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.055+0000 7f38a6ffd700 1 -- 172.21.15.17:0/2450787933 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3898004160 con 0x7f38a807bde0 2023-07-14T11:53:12.088 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.055+0000 7f38a6ffd700 1 -- 172.21.15.17:0/2450787933 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38980042c0 con 0x7f38a807bde0 2023-07-14T11:53:12.088 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.055+0000 7f38a6ffd700 1 -- 172.21.15.17:0/2450787933 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f38980119b0 con 0x7f38a807bde0 2023-07-14T11:53:12.089 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.055+0000 7f38ae435700 1 -- 172.21.15.17:0/2450787933 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38a807bde0 msgr2=0x7f38a807a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:12.089 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.055+0000 7f38ae435700 1 --2- 172.21.15.17:0/2450787933 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38a807bde0 0x7f38a807a2e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f38980098a0 tx=0x7f3898009bb0 comp rx=0 tx=0).stop 2023-07-14T11:53:12.089 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.055+0000 7f38ae435700 1 -- 172.21.15.17:0/2450787933 shutdown_connections 2023-07-14T11:53:12.089 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.055+0000 7f38ae435700 1 --2- 172.21.15.17:0/2450787933 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38a807bde0 0x7f38a807a2e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:12.089 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.055+0000 7f38ae435700 1 -- 172.21.15.17:0/2450787933 >> 172.21.15.17:0/2450787933 conn(0x7f38a8100970 msgr2=0x7f38a8102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:12.089 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.055+0000 7f38ae435700 1 -- 172.21.15.17:0/2450787933 shutdown_connections 2023-07-14T11:53:12.089 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.056+0000 7f38ae435700 1 -- 172.21.15.17:0/2450787933 wait complete. 2023-07-14T11:53:12.090 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.056+0000 7f38ae435700 1 Processor -- start 2023-07-14T11:53:12.090 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.057+0000 7f38ae435700 1 -- start start 2023-07-14T11:53:12.090 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.057+0000 7f38ae435700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38a807bde0 0x7f38a819b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:12.090 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.057+0000 7f38ae435700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a819b860 con 0x7f38a807bde0 2023-07-14T11:53:12.090 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.057+0000 7f38a7fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38a807bde0 0x7f38a819b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:12.090 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.057+0000 7f38a7fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38a807bde0 0x7f38a819b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49952/0 (socket says 172.21.15.17:49952) 2023-07-14T11:53:12.090 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.057+0000 7f38a7fff700 1 -- 172.21.15.17:0/504559577 learned_addr learned my addr 172.21.15.17:0/504559577 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:53:12.090 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.057+0000 7f38a7fff700 1 -- 172.21.15.17:0/504559577 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3898009550 con 0x7f38a807bde0 2023-07-14T11:53:12.091 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.058+0000 7f38a7fff700 1 --2- 172.21.15.17:0/504559577 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38a807bde0 0x7f38a819b320 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3898003e60 tx=0x7f3898011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:12.091 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.058+0000 7f38a57fa700 1 -- 172.21.15.17:0/504559577 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3898003a60 con 0x7f38a807bde0 2023-07-14T11:53:12.091 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.058+0000 7f38a57fa700 1 -- 172.21.15.17:0/504559577 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3898003bc0 con 0x7f38a807bde0 2023-07-14T11:53:12.091 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.058+0000 7f38ae435700 1 -- 172.21.15.17:0/504559577 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a819ba60 con 0x7f38a807bde0 2023-07-14T11:53:12.091 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.058+0000 7f38a57fa700 1 -- 172.21.15.17:0/504559577 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f38980196c0 con 0x7f38a807bde0 2023-07-14T11:53:12.091 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.058+0000 7f38ae435700 1 -- 172.21.15.17:0/504559577 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a819be80 con 0x7f38a807bde0 2023-07-14T11:53:12.091 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.059+0000 7f38a57fa700 1 -- 172.21.15.17:0/504559577 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f389801ab60 con 0x7f38a807bde0 2023-07-14T11:53:12.091 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.059+0000 7f38ae435700 1 -- 172.21.15.17:0/504559577 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38a8195300 con 0x7f38a807bde0 2023-07-14T11:53:12.092 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.060+0000 7f38a57fa700 1 --2- 172.21.15.17:0/504559577 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3890038790 0x7f389003ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:12.092 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.060+0000 7f38a57fa700 1 -- 172.21.15.17:0/504559577 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f389804cae0 con 0x7f38a807bde0 2023-07-14T11:53:12.092 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.060+0000 7f38a77fe700 1 --2- 172.21.15.17:0/504559577 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3890038790 0x7f389003ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:12.093 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.060+0000 7f38a77fe700 1 --2- 172.21.15.17:0/504559577 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3890038790 0x7f389003ac40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f389c006fd0 tx=0x7f389c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:12.093 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.064+0000 7f38a57fa700 1 -- 172.21.15.17:0/504559577 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3898018b00 con 0x7f38a807bde0 2023-07-14T11:53:12.094 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.280+0000 7f38ae435700 1 -- 172.21.15.17:0/504559577 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f38a802cce0 con 0x7f38a807bde0 2023-07-14T11:53:12.094 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.280+0000 7f38a57fa700 1 -- 172.21.15.17:0/504559577 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f3898018b00 con 0x7f38a807bde0 2023-07-14T11:53:12.094 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.283+0000 7f38ae435700 1 -- 172.21.15.17:0/504559577 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3890038790 msgr2=0x7f389003ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:12.094 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.283+0000 7f38ae435700 1 --2- 172.21.15.17:0/504559577 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3890038790 0x7f389003ac40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f389c006fd0 tx=0x7f389c006e40 comp rx=0 tx=0).stop 2023-07-14T11:53:12.094 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.283+0000 7f38ae435700 1 -- 172.21.15.17:0/504559577 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38a807bde0 msgr2=0x7f38a819b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:12.094 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.283+0000 7f38ae435700 1 --2- 172.21.15.17:0/504559577 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38a807bde0 0x7f38a819b320 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3898003e60 tx=0x7f3898011a30 comp rx=0 tx=0).stop 2023-07-14T11:53:12.094 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.283+0000 7f38ae435700 1 -- 172.21.15.17:0/504559577 shutdown_connections 2023-07-14T11:53:12.094 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.283+0000 7f38ae435700 1 --2- 172.21.15.17:0/504559577 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3890038790 0x7f389003ac40 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:12.095 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.283+0000 7f38ae435700 1 --2- 172.21.15.17:0/504559577 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38a807bde0 0x7f38a819b320 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:12.095 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.283+0000 7f38ae435700 1 -- 172.21.15.17:0/504559577 >> 172.21.15.17:0/504559577 conn(0x7f38a8100970 msgr2=0x7f38a8102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:12.095 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.283+0000 7f38ae435700 1 -- 172.21.15.17:0/504559577 shutdown_connections 2023-07-14T11:53:12.095 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:11.283+0000 7f38ae435700 1 -- 172.21.15.17:0/504559577 wait complete. 2023-07-14T11:53:12.100 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:11 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/504559577' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-07-14T11:53:12.109 INFO:teuthology.orchestra.run.smithi017.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-14T11:53:12.109 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stdout disabled 2023-07-14T11:53:12.126 INFO:teuthology.orchestra.run.smithi017.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-14T11:53:12.126 INFO:teuthology.orchestra.run.smithi017.stdout:systemctl: stdout inactive 2023-07-14T11:53:12.126 INFO:teuthology.orchestra.run.smithi017.stdout:firewalld.service is not enabled 2023-07-14T11:53:12.126 INFO:teuthology.orchestra.run.smithi017.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-07-14T11:53:12.129 INFO:teuthology.orchestra.run.smithi017.stdout:Ceph Dashboard is now available at: 2023-07-14T11:53:12.129 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:53:12.129 INFO:teuthology.orchestra.run.smithi017.stdout: URL: https://smithi017.front.sepia.ceph.com:8443/ 2023-07-14T11:53:12.129 INFO:teuthology.orchestra.run.smithi017.stdout: User: admin 2023-07-14T11:53:12.130 INFO:teuthology.orchestra.run.smithi017.stdout: Password: rvafxc4u1s 2023-07-14T11:53:12.130 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:53:12.130 INFO:teuthology.orchestra.run.smithi017.stdout:Saving cluster configuration to /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/config directory 2023-07-14T11:53:12.130 INFO:teuthology.orchestra.run.smithi017.stdout:Enabling autotune for osd_memory_target 2023-07-14T11:53:13.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:12 smithi017 ceph-mon[126339]: mgrmap e13: x(active, since 6s) 2023-07-14T11:53:13.512 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.716+0000 7f23ead5f700 1 Processor -- start 2023-07-14T11:53:13.512 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.716+0000 7f23ead5f700 1 -- start start 2023-07-14T11:53:13.512 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.716+0000 7f23ead5f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23e407be20 0x7f23e407a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:13.512 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.716+0000 7f23ead5f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e407a860 con 0x7f23e407be20 2023-07-14T11:53:13.513 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.717+0000 7f23e8afb700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23e407be20 0x7f23e407a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:13.513 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.717+0000 7f23e8afb700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23e407be20 0x7f23e407a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49954/0 (socket says 172.21.15.17:49954) 2023-07-14T11:53:13.513 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.717+0000 7f23e8afb700 1 -- 172.21.15.17:0/3116763428 learned_addr learned my addr 172.21.15.17:0/3116763428 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:53:13.513 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.718+0000 7f23e8afb700 1 -- 172.21.15.17:0/3116763428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e407a9a0 con 0x7f23e407be20 2023-07-14T11:53:13.513 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.718+0000 7f23e8afb700 1 --2- 172.21.15.17:0/3116763428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23e407be20 0x7f23e407a320 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f23d80098a0 tx=0x7f23d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53770a418aa9d7a5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:13.513 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.718+0000 7f23e37fe700 1 -- 172.21.15.17:0/3116763428 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f23d8004160 con 0x7f23e407be20 2023-07-14T11:53:13.513 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.718+0000 7f23e37fe700 1 -- 172.21.15.17:0/3116763428 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23d80042c0 con 0x7f23e407be20 2023-07-14T11:53:13.514 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.718+0000 7f23e37fe700 1 -- 172.21.15.17:0/3116763428 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f23d80119b0 con 0x7f23e407be20 2023-07-14T11:53:13.514 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.719+0000 7f23ead5f700 1 -- 172.21.15.17:0/3116763428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23e407be20 msgr2=0x7f23e407a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:13.514 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.719+0000 7f23ead5f700 1 --2- 172.21.15.17:0/3116763428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23e407be20 0x7f23e407a320 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f23d80098a0 tx=0x7f23d8009bb0 comp rx=0 tx=0).stop 2023-07-14T11:53:13.514 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.719+0000 7f23ead5f700 1 -- 172.21.15.17:0/3116763428 shutdown_connections 2023-07-14T11:53:13.514 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.719+0000 7f23ead5f700 1 --2- 172.21.15.17:0/3116763428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23e407be20 0x7f23e407a320 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:13.514 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.719+0000 7f23ead5f700 1 -- 172.21.15.17:0/3116763428 >> 172.21.15.17:0/3116763428 conn(0x7f23e4100970 msgr2=0x7f23e4102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:13.514 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.719+0000 7f23ead5f700 1 -- 172.21.15.17:0/3116763428 shutdown_connections 2023-07-14T11:53:13.514 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.719+0000 7f23ead5f700 1 -- 172.21.15.17:0/3116763428 wait complete. 2023-07-14T11:53:13.515 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.720+0000 7f23ead5f700 1 Processor -- start 2023-07-14T11:53:13.515 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.720+0000 7f23ead5f700 1 -- start start 2023-07-14T11:53:13.515 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.720+0000 7f23ead5f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23e407be20 0x7f23e419b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:13.515 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.720+0000 7f23ead5f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e419b890 con 0x7f23e407be20 2023-07-14T11:53:13.515 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.721+0000 7f23e8afb700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23e407be20 0x7f23e419b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:13.515 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.721+0000 7f23e8afb700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23e407be20 0x7f23e419b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49970/0 (socket says 172.21.15.17:49970) 2023-07-14T11:53:13.515 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.721+0000 7f23e8afb700 1 -- 172.21.15.17:0/1376884344 learned_addr learned my addr 172.21.15.17:0/1376884344 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:53:13.515 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.721+0000 7f23e8afb700 1 -- 172.21.15.17:0/1376884344 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d8009550 con 0x7f23e407be20 2023-07-14T11:53:13.516 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.721+0000 7f23e8afb700 1 --2- 172.21.15.17:0/1376884344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23e407be20 0x7f23e419b350 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f23d8000c00 tx=0x7f23d8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:13.516 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.721+0000 7f23e1ffb700 1 -- 172.21.15.17:0/1376884344 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f23d8003aa0 con 0x7f23e407be20 2023-07-14T11:53:13.516 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.721+0000 7f23e1ffb700 1 -- 172.21.15.17:0/1376884344 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23d8003c00 con 0x7f23e407be20 2023-07-14T11:53:13.516 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.721+0000 7f23ead5f700 1 -- 172.21.15.17:0/1376884344 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23e419ba90 con 0x7f23e407be20 2023-07-14T11:53:13.516 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.721+0000 7f23e1ffb700 1 -- 172.21.15.17:0/1376884344 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f23d8024800 con 0x7f23e407be20 2023-07-14T11:53:13.516 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.722+0000 7f23ead5f700 1 -- 172.21.15.17:0/1376884344 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23e419beb0 con 0x7f23e407be20 2023-07-14T11:53:13.516 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.722+0000 7f23e1ffb700 1 -- 172.21.15.17:0/1376884344 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f23d801ab60 con 0x7f23e407be20 2023-07-14T11:53:13.517 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.722+0000 7f23ead5f700 1 -- 172.21.15.17:0/1376884344 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23e4195330 con 0x7f23e407be20 2023-07-14T11:53:13.517 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.723+0000 7f23e1ffb700 1 --2- 172.21.15.17:0/1376884344 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f23d4038850 0x7f23d403ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:13.517 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.723+0000 7f23e1ffb700 1 -- 172.21.15.17:0/1376884344 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f23d804ca10 con 0x7f23e407be20 2023-07-14T11:53:13.517 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.723+0000 7f23e3fff700 1 --2- 172.21.15.17:0/1376884344 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f23d4038850 0x7f23d403ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:13.517 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.724+0000 7f23e3fff700 1 --2- 172.21.15.17:0/1376884344 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f23d4038850 0x7f23d403ad00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f23d0006fd0 tx=0x7f23d0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:13.517 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.728+0000 7f23e1ffb700 1 -- 172.21.15.17:0/1376884344 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f23d8018530 con 0x7f23e407be20 2023-07-14T11:53:13.517 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.933+0000 7f23ead5f700 1 -- 172.21.15.17:0/1376884344 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f23e4061b40 con 0x7f23e407be20 2023-07-14T11:53:13.517 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.934+0000 7f23e1ffb700 1 -- 172.21.15.17:0/1376884344 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f23d8018350 con 0x7f23e407be20 2023-07-14T11:53:13.518 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.936+0000 7f23ead5f700 1 -- 172.21.15.17:0/1376884344 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f23d4038850 msgr2=0x7f23d403ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:13.518 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.936+0000 7f23ead5f700 1 --2- 172.21.15.17:0/1376884344 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f23d4038850 0x7f23d403ad00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f23d0006fd0 tx=0x7f23d0006e40 comp rx=0 tx=0).stop 2023-07-14T11:53:13.518 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.936+0000 7f23ead5f700 1 -- 172.21.15.17:0/1376884344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23e407be20 msgr2=0x7f23e419b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:13.518 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.936+0000 7f23ead5f700 1 --2- 172.21.15.17:0/1376884344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23e407be20 0x7f23e419b350 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f23d8000c00 tx=0x7f23d8011a30 comp rx=0 tx=0).stop 2023-07-14T11:53:13.518 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.936+0000 7f23ead5f700 1 -- 172.21.15.17:0/1376884344 shutdown_connections 2023-07-14T11:53:13.518 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.936+0000 7f23ead5f700 1 --2- 172.21.15.17:0/1376884344 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f23d4038850 0x7f23d403ad00 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:13.518 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.936+0000 7f23ead5f700 1 --2- 172.21.15.17:0/1376884344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23e407be20 0x7f23e419b350 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:13.519 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.936+0000 7f23ead5f700 1 -- 172.21.15.17:0/1376884344 >> 172.21.15.17:0/1376884344 conn(0x7f23e4100970 msgr2=0x7f23e4102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:13.519 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.936+0000 7f23ead5f700 1 -- 172.21.15.17:0/1376884344 shutdown_connections 2023-07-14T11:53:13.519 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:12.936+0000 7f23ead5f700 1 -- 172.21.15.17:0/1376884344 wait complete. 2023-07-14T11:53:14.897 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.068+0000 7f4a1a7c8700 1 Processor -- start 2023-07-14T11:53:14.897 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.068+0000 7f4a1a7c8700 1 -- start start 2023-07-14T11:53:14.897 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.069+0000 7f4a1a7c8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a1407be40 0x7f4a1407a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:14.898 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.069+0000 7f4a1a7c8700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1407a880 con 0x7f4a1407be40 2023-07-14T11:53:14.898 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.069+0000 7f4a13fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a1407be40 0x7f4a1407a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:14.898 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.069+0000 7f4a13fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a1407be40 0x7f4a1407a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59592/0 (socket says 172.21.15.17:59592) 2023-07-14T11:53:14.898 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.069+0000 7f4a13fff700 1 -- 172.21.15.17:0/2279802608 learned_addr learned my addr 172.21.15.17:0/2279802608 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:53:14.898 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.069+0000 7f4a13fff700 1 -- 172.21.15.17:0/2279802608 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a1407a9c0 con 0x7f4a1407be40 2023-07-14T11:53:14.898 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.070+0000 7f4a13fff700 1 --2- 172.21.15.17:0/2279802608 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a1407be40 0x7f4a1407a340 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4a040098a0 tx=0x7f4a04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73202d6a3e8c5e9f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:14.898 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.070+0000 7f4a12ffd700 1 -- 172.21.15.17:0/2279802608 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4a04004160 con 0x7f4a1407be40 2023-07-14T11:53:14.898 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.070+0000 7f4a12ffd700 1 -- 172.21.15.17:0/2279802608 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a040042c0 con 0x7f4a1407be40 2023-07-14T11:53:14.899 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.070+0000 7f4a12ffd700 1 -- 172.21.15.17:0/2279802608 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4a040119b0 con 0x7f4a1407be40 2023-07-14T11:53:14.899 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.070+0000 7f4a1a7c8700 1 -- 172.21.15.17:0/2279802608 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a1407be40 msgr2=0x7f4a1407a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:14.899 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.070+0000 7f4a1a7c8700 1 --2- 172.21.15.17:0/2279802608 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a1407be40 0x7f4a1407a340 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4a040098a0 tx=0x7f4a04009bb0 comp rx=0 tx=0).stop 2023-07-14T11:53:14.899 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.071+0000 7f4a1a7c8700 1 -- 172.21.15.17:0/2279802608 shutdown_connections 2023-07-14T11:53:14.899 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.071+0000 7f4a1a7c8700 1 --2- 172.21.15.17:0/2279802608 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a1407be40 0x7f4a1407a340 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:14.899 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.071+0000 7f4a1a7c8700 1 -- 172.21.15.17:0/2279802608 >> 172.21.15.17:0/2279802608 conn(0x7f4a14100970 msgr2=0x7f4a14102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:14.899 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.071+0000 7f4a1a7c8700 1 -- 172.21.15.17:0/2279802608 shutdown_connections 2023-07-14T11:53:14.900 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.071+0000 7f4a1a7c8700 1 -- 172.21.15.17:0/2279802608 wait complete. 2023-07-14T11:53:14.900 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.072+0000 7f4a1a7c8700 1 Processor -- start 2023-07-14T11:53:14.900 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.072+0000 7f4a1a7c8700 1 -- start start 2023-07-14T11:53:14.900 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.072+0000 7f4a1a7c8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a1407be40 0x7f4a1419b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:14.900 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.072+0000 7f4a1a7c8700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1419b870 con 0x7f4a1407be40 2023-07-14T11:53:14.900 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.072+0000 7f4a13fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a1407be40 0x7f4a1419b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:14.900 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.072+0000 7f4a13fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a1407be40 0x7f4a1419b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59600/0 (socket says 172.21.15.17:59600) 2023-07-14T11:53:14.900 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.072+0000 7f4a13fff700 1 -- 172.21.15.17:0/244973048 learned_addr learned my addr 172.21.15.17:0/244973048 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:53:14.901 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.073+0000 7f4a13fff700 1 -- 172.21.15.17:0/244973048 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a04009550 con 0x7f4a1407be40 2023-07-14T11:53:14.901 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.073+0000 7f4a13fff700 1 --2- 172.21.15.17:0/244973048 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a1407be40 0x7f4a1419b330 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4a04003e60 tx=0x7f4a04011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:14.901 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.073+0000 7f4a117fa700 1 -- 172.21.15.17:0/244973048 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4a04003af0 con 0x7f4a1407be40 2023-07-14T11:53:14.901 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.073+0000 7f4a117fa700 1 -- 172.21.15.17:0/244973048 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a04003c50 con 0x7f4a1407be40 2023-07-14T11:53:14.901 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.073+0000 7f4a1a7c8700 1 -- 172.21.15.17:0/244973048 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a1419ba70 con 0x7f4a1407be40 2023-07-14T11:53:14.901 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.073+0000 7f4a117fa700 1 -- 172.21.15.17:0/244973048 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4a04019790 con 0x7f4a1407be40 2023-07-14T11:53:14.901 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.073+0000 7f4a1a7c8700 1 -- 172.21.15.17:0/244973048 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a1419be90 con 0x7f4a1407be40 2023-07-14T11:53:14.902 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.074+0000 7f4a117fa700 1 -- 172.21.15.17:0/244973048 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f4a0401abd0 con 0x7f4a1407be40 2023-07-14T11:53:14.902 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.074+0000 7f4a1a7c8700 1 -- 172.21.15.17:0/244973048 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a14195310 con 0x7f4a1407be40 2023-07-14T11:53:14.902 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.075+0000 7f4a117fa700 1 --2- 172.21.15.17:0/244973048 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f49fc038800 0x7f49fc03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:14.902 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.075+0000 7f4a117fa700 1 -- 172.21.15.17:0/244973048 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4a0404cc90 con 0x7f4a1407be40 2023-07-14T11:53:14.902 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.075+0000 7f4a137fe700 1 --2- 172.21.15.17:0/244973048 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f49fc038800 0x7f49fc03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:14.902 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.076+0000 7f4a137fe700 1 --2- 172.21.15.17:0/244973048 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f49fc038800 0x7f49fc03acb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4a08006fd0 tx=0x7f4a08006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:14.902 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.080+0000 7f4a117fa700 1 -- 172.21.15.17:0/244973048 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4a04010790 con 0x7f4a1407be40 2023-07-14T11:53:14.902 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.397+0000 7f4a1a7c8700 1 -- 172.21.15.17:0/244973048 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f4a1402cce0 con 0x7f4a1407be40 2023-07-14T11:53:14.903 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.399+0000 7f4a117fa700 1 -- 172.21.15.17:0/244973048 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f4a040281c0 con 0x7f4a1407be40 2023-07-14T11:53:14.903 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.402+0000 7f4a1a7c8700 1 -- 172.21.15.17:0/244973048 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f49fc038800 msgr2=0x7f49fc03acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:14.903 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.402+0000 7f4a1a7c8700 1 --2- 172.21.15.17:0/244973048 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f49fc038800 0x7f49fc03acb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4a08006fd0 tx=0x7f4a08006e40 comp rx=0 tx=0).stop 2023-07-14T11:53:14.903 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.402+0000 7f4a1a7c8700 1 -- 172.21.15.17:0/244973048 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a1407be40 msgr2=0x7f4a1419b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:14.903 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.402+0000 7f4a1a7c8700 1 --2- 172.21.15.17:0/244973048 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a1407be40 0x7f4a1419b330 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4a04003e60 tx=0x7f4a04011aa0 comp rx=0 tx=0).stop 2023-07-14T11:53:14.903 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.402+0000 7f4a1a7c8700 1 -- 172.21.15.17:0/244973048 shutdown_connections 2023-07-14T11:53:14.903 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.402+0000 7f4a1a7c8700 1 --2- 172.21.15.17:0/244973048 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f49fc038800 0x7f49fc03acb0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:14.903 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.402+0000 7f4a1a7c8700 1 --2- 172.21.15.17:0/244973048 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a1407be40 0x7f4a1419b330 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:14.904 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.402+0000 7f4a1a7c8700 1 -- 172.21.15.17:0/244973048 >> 172.21.15.17:0/244973048 conn(0x7f4a14100970 msgr2=0x7f4a14102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:14.904 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.402+0000 7f4a1a7c8700 1 -- 172.21.15.17:0/244973048 shutdown_connections 2023-07-14T11:53:14.904 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr 2023-07-14T11:53:14.402+0000 7f4a1a7c8700 1 -- 172.21.15.17:0/244973048 wait complete. 2023-07-14T11:53:14.904 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-07-14T11:53:14.904 INFO:teuthology.orchestra.run.smithi017.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-07-14T11:53:14.904 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:53:14.904 INFO:teuthology.orchestra.run.smithi017.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-07-14T11:53:14.905 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:53:14.905 INFO:teuthology.orchestra.run.smithi017.stdout:Or, if you are only running a single cluster on this host: 2023-07-14T11:53:14.905 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:53:14.905 INFO:teuthology.orchestra.run.smithi017.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-07-14T11:53:14.905 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:53:14.905 INFO:teuthology.orchestra.run.smithi017.stdout:Please consider enabling telemetry to help improve Ceph: 2023-07-14T11:53:14.905 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:53:14.905 INFO:teuthology.orchestra.run.smithi017.stdout: ceph telemetry on 2023-07-14T11:53:14.906 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:53:14.906 INFO:teuthology.orchestra.run.smithi017.stdout:For more information see: 2023-07-14T11:53:14.906 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:53:14.906 INFO:teuthology.orchestra.run.smithi017.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-07-14T11:53:14.906 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:53:14.906 INFO:teuthology.orchestra.run.smithi017.stdout:Bootstrap complete. 2023-07-14T11:53:14.955 INFO:tasks.cephadm:Fetching config... 2023-07-14T11:53:14.955 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2023-07-14T11:53:14.956 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-07-14T11:53:15.001 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-07-14T11:53:15.002 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2023-07-14T11:53:15.002 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-07-14T11:53:15.087 INFO:tasks.cephadm:Fetching mon keyring... 2023-07-14T11:53:15.087 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2023-07-14T11:53:15.087 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/keyring of=/dev/stdout 2023-07-14T11:53:15.198 INFO:tasks.cephadm:Fetching pub ssh key... 2023-07-14T11:53:15.198 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2023-07-14T11:53:15.198 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-07-14T11:53:15.281 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-07-14T11:53:15.282 DEBUG:teuthology.orchestra.run.smithi017:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHEDJqnq89C2tfGNhua+JBX8WLo8suZasySMTCYKhU7YrirHfsweZzISghGTSXhEHSWZKTebQ2Frr1nZjlr8KxyKwVb1etzJVQL16ZBHiEcIawZj9Wbvv8BX6zBcRP3LG1/S4onylYhkWkuow91spLrISSqQL2EdjcRAgepPs0KV39gVRHbVDQ2Y3syjoZ9xGqCGkHg+uJu6fJupm/oD42FHpB7zgq6LKsa/fYLpR6KDbXXFVjMlrQp8sjTtOa+rSuN5LPcXeGpGVgKv9IFHTE01USoyFqFLM80UHCqHJU5wXJxJB368PX6noJpiFoYVI+GV0orxNJiQnrVTlyTzldUQaXcUZ3quPKctmCejVl0cEIl7hlO3z8PV3LucxA42gg/Zuk6qz0qrd82gCP5FXE9yBxzpVMIgZe1hIufu3/5Vk59KmXa5iGZaNbLPB8YKKotSOPFONpGuSpybuPvJvmk1SqXhmVNKgEqo5vRfZdGEt/nmejbcKyKAJXn2hfrvE= ceph-b3c386cc-223c-11ee-9b33-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-07-14T11:53:15.427 INFO:teuthology.orchestra.run.smithi017.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHEDJqnq89C2tfGNhua+JBX8WLo8suZasySMTCYKhU7YrirHfsweZzISghGTSXhEHSWZKTebQ2Frr1nZjlr8KxyKwVb1etzJVQL16ZBHiEcIawZj9Wbvv8BX6zBcRP3LG1/S4onylYhkWkuow91spLrISSqQL2EdjcRAgepPs0KV39gVRHbVDQ2Y3syjoZ9xGqCGkHg+uJu6fJupm/oD42FHpB7zgq6LKsa/fYLpR6KDbXXFVjMlrQp8sjTtOa+rSuN5LPcXeGpGVgKv9IFHTE01USoyFqFLM80UHCqHJU5wXJxJB368PX6noJpiFoYVI+GV0orxNJiQnrVTlyTzldUQaXcUZ3quPKctmCejVl0cEIl7hlO3z8PV3LucxA42gg/Zuk6qz0qrd82gCP5FXE9yBxzpVMIgZe1hIufu3/5Vk59KmXa5iGZaNbLPB8YKKotSOPFONpGuSpybuPvJvmk1SqXhmVNKgEqo5vRfZdGEt/nmejbcKyKAJXn2hfrvE= ceph-b3c386cc-223c-11ee-9b33-001a4aab830c 2023-07-14T11:53:15.450 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:15 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/244973048' entity='client.admin' 2023-07-14T11:53:15.461 DEBUG:teuthology.orchestra.run.smithi093:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHEDJqnq89C2tfGNhua+JBX8WLo8suZasySMTCYKhU7YrirHfsweZzISghGTSXhEHSWZKTebQ2Frr1nZjlr8KxyKwVb1etzJVQL16ZBHiEcIawZj9Wbvv8BX6zBcRP3LG1/S4onylYhkWkuow91spLrISSqQL2EdjcRAgepPs0KV39gVRHbVDQ2Y3syjoZ9xGqCGkHg+uJu6fJupm/oD42FHpB7zgq6LKsa/fYLpR6KDbXXFVjMlrQp8sjTtOa+rSuN5LPcXeGpGVgKv9IFHTE01USoyFqFLM80UHCqHJU5wXJxJB368PX6noJpiFoYVI+GV0orxNJiQnrVTlyTzldUQaXcUZ3quPKctmCejVl0cEIl7hlO3z8PV3LucxA42gg/Zuk6qz0qrd82gCP5FXE9yBxzpVMIgZe1hIufu3/5Vk59KmXa5iGZaNbLPB8YKKotSOPFONpGuSpybuPvJvmk1SqXhmVNKgEqo5vRfZdGEt/nmejbcKyKAJXn2hfrvE= ceph-b3c386cc-223c-11ee-9b33-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-07-14T11:53:15.507 INFO:teuthology.orchestra.run.smithi093.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHEDJqnq89C2tfGNhua+JBX8WLo8suZasySMTCYKhU7YrirHfsweZzISghGTSXhEHSWZKTebQ2Frr1nZjlr8KxyKwVb1etzJVQL16ZBHiEcIawZj9Wbvv8BX6zBcRP3LG1/S4onylYhkWkuow91spLrISSqQL2EdjcRAgepPs0KV39gVRHbVDQ2Y3syjoZ9xGqCGkHg+uJu6fJupm/oD42FHpB7zgq6LKsa/fYLpR6KDbXXFVjMlrQp8sjTtOa+rSuN5LPcXeGpGVgKv9IFHTE01USoyFqFLM80UHCqHJU5wXJxJB368PX6noJpiFoYVI+GV0orxNJiQnrVTlyTzldUQaXcUZ3quPKctmCejVl0cEIl7hlO3z8PV3LucxA42gg/Zuk6qz0qrd82gCP5FXE9yBxzpVMIgZe1hIufu3/5Vk59KmXa5iGZaNbLPB8YKKotSOPFONpGuSpybuPvJvmk1SqXhmVNKgEqo5vRfZdGEt/nmejbcKyKAJXn2hfrvE= ceph-b3c386cc-223c-11ee-9b33-001a4aab830c 2023-07-14T11:53:15.520 DEBUG:teuthology.orchestra.run.smithi097:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHEDJqnq89C2tfGNhua+JBX8WLo8suZasySMTCYKhU7YrirHfsweZzISghGTSXhEHSWZKTebQ2Frr1nZjlr8KxyKwVb1etzJVQL16ZBHiEcIawZj9Wbvv8BX6zBcRP3LG1/S4onylYhkWkuow91spLrISSqQL2EdjcRAgepPs0KV39gVRHbVDQ2Y3syjoZ9xGqCGkHg+uJu6fJupm/oD42FHpB7zgq6LKsa/fYLpR6KDbXXFVjMlrQp8sjTtOa+rSuN5LPcXeGpGVgKv9IFHTE01USoyFqFLM80UHCqHJU5wXJxJB368PX6noJpiFoYVI+GV0orxNJiQnrVTlyTzldUQaXcUZ3quPKctmCejVl0cEIl7hlO3z8PV3LucxA42gg/Zuk6qz0qrd82gCP5FXE9yBxzpVMIgZe1hIufu3/5Vk59KmXa5iGZaNbLPB8YKKotSOPFONpGuSpybuPvJvmk1SqXhmVNKgEqo5vRfZdGEt/nmejbcKyKAJXn2hfrvE= ceph-b3c386cc-223c-11ee-9b33-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-07-14T11:53:15.564 INFO:teuthology.orchestra.run.smithi097.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHEDJqnq89C2tfGNhua+JBX8WLo8suZasySMTCYKhU7YrirHfsweZzISghGTSXhEHSWZKTebQ2Frr1nZjlr8KxyKwVb1etzJVQL16ZBHiEcIawZj9Wbvv8BX6zBcRP3LG1/S4onylYhkWkuow91spLrISSqQL2EdjcRAgepPs0KV39gVRHbVDQ2Y3syjoZ9xGqCGkHg+uJu6fJupm/oD42FHpB7zgq6LKsa/fYLpR6KDbXXFVjMlrQp8sjTtOa+rSuN5LPcXeGpGVgKv9IFHTE01USoyFqFLM80UHCqHJU5wXJxJB368PX6noJpiFoYVI+GV0orxNJiQnrVTlyTzldUQaXcUZ3quPKctmCejVl0cEIl7hlO3z8PV3LucxA42gg/Zuk6qz0qrd82gCP5FXE9yBxzpVMIgZe1hIufu3/5Vk59KmXa5iGZaNbLPB8YKKotSOPFONpGuSpybuPvJvmk1SqXhmVNKgEqo5vRfZdGEt/nmejbcKyKAJXn2hfrvE= ceph-b3c386cc-223c-11ee-9b33-001a4aab830c 2023-07-14T11:53:15.577 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-07-14T11:53:16.582 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T11:53:18.725 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:18.741+0000 7f6d09b1d700 1 -- 172.21.15.17:0/2014453788 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d040ff5e0 msgr2=0x7f6d040ff9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:18.725 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:18.741+0000 7f6d09b1d700 1 --2- 172.21.15.17:0/2014453788 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d040ff5e0 0x7f6d040ff9f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4009910 tx=0x7f6cf4009c20 comp rx=0 tx=0).stop 2023-07-14T11:53:18.726 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:18.741+0000 7f6d09b1d700 1 -- 172.21.15.17:0/2014453788 shutdown_connections 2023-07-14T11:53:18.726 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:18.741+0000 7f6d09b1d700 1 --2- 172.21.15.17:0/2014453788 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d040ff5e0 0x7f6d040ff9f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:18.726 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:18.741+0000 7f6d09b1d700 1 -- 172.21.15.17:0/2014453788 >> 172.21.15.17:0/2014453788 conn(0x7f6d040fab90 msgr2=0x7f6d040fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:18.726 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:18.741+0000 7f6d09b1d700 1 -- 172.21.15.17:0/2014453788 shutdown_connections 2023-07-14T11:53:18.726 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:18.741+0000 7f6d09b1d700 1 -- 172.21.15.17:0/2014453788 wait complete. 2023-07-14T11:53:18.727 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:18.742+0000 7f6d09b1d700 1 Processor -- start 2023-07-14T11:53:18.727 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:18.742+0000 7f6d09b1d700 1 -- start start 2023-07-14T11:53:18.727 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:18.743+0000 7f6d09b1d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d040ff5e0 0x7f6d04194750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:18.728 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:18.743+0000 7f6d09b1d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d04194c90 con 0x7f6d040ff5e0 2023-07-14T11:53:18.728 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:18.743+0000 7f6d037fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d040ff5e0 0x7f6d04194750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:18.728 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:18.743+0000 7f6d037fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d040ff5e0 0x7f6d04194750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59612/0 (socket says 172.21.15.17:59612) 2023-07-14T11:53:18.728 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:18.743+0000 7f6d037fe700 1 -- 172.21.15.17:0/2565524263 learned_addr learned my addr 172.21.15.17:0/2565524263 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:53:18.728 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:18.743+0000 7f6d037fe700 1 -- 172.21.15.17:0/2565524263 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf40095f0 con 0x7f6d040ff5e0 2023-07-14T11:53:18.728 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:18.744+0000 7f6d037fe700 1 --2- 172.21.15.17:0/2565524263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d040ff5e0 0x7f6d04194750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4004f40 tx=0x7f6cf40057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:18.729 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:18.744+0000 7f6d00ff9700 1 -- 172.21.15.17:0/2565524263 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6cf401d070 con 0x7f6d040ff5e0 2023-07-14T11:53:18.729 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:18.744+0000 7f6d00ff9700 1 -- 172.21.15.17:0/2565524263 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6cf400bc10 con 0x7f6d040ff5e0 2023-07-14T11:53:18.729 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:18.744+0000 7f6d09b1d700 1 -- 172.21.15.17:0/2565524263 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d04194e90 con 0x7f6d040ff5e0 2023-07-14T11:53:18.729 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:18.744+0000 7f6d00ff9700 1 -- 172.21.15.17:0/2565524263 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6cf4003a70 con 0x7f6d040ff5e0 2023-07-14T11:53:18.729 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:18.745+0000 7f6d09b1d700 1 -- 172.21.15.17:0/2565524263 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d04195330 con 0x7f6d040ff5e0 2023-07-14T11:53:18.730 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:18.746+0000 7f6d09b1d700 1 -- 172.21.15.17:0/2565524263 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d0418e1b0 con 0x7f6d040ff5e0 2023-07-14T11:53:18.731 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:18.747+0000 7f6d00ff9700 1 -- 172.21.15.17:0/2565524263 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f6cf4003ce0 con 0x7f6d040ff5e0 2023-07-14T11:53:18.731 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:18.748+0000 7f6d00ff9700 1 --2- 172.21.15.17:0/2565524263 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f6cec038850 0x7f6cec03ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:18.731 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:18.748+0000 7f6d00ff9700 1 -- 172.21.15.17:0/2565524263 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6cf404c9b0 con 0x7f6d040ff5e0 2023-07-14T11:53:18.732 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:18.748+0000 7f6d02ffd700 1 --2- 172.21.15.17:0/2565524263 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f6cec038850 0x7f6cec03ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:18.732 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:18.749+0000 7f6d02ffd700 1 --2- 172.21.15.17:0/2565524263 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f6cec038850 0x7f6cec03ad00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8006fd0 tx=0x7f6cf8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:18.736 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:18.752+0000 7f6d00ff9700 1 -- 172.21.15.17:0/2565524263 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6cf4021bb0 con 0x7f6d040ff5e0 2023-07-14T11:53:18.944 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:18.960+0000 7f6d09b1d700 1 -- 172.21.15.17:0/2565524263 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f6d0402c850 con 0x7f6d040ff5e0 2023-07-14T11:53:18.947 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:18.963+0000 7f6d00ff9700 1 -- 172.21.15.17:0/2565524263 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f6cf4026030 con 0x7f6d040ff5e0 2023-07-14T11:53:18.950 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:18.966+0000 7f6d09b1d700 1 -- 172.21.15.17:0/2565524263 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f6cec038850 msgr2=0x7f6cec03ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:18.950 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:18.966+0000 7f6d09b1d700 1 --2- 172.21.15.17:0/2565524263 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f6cec038850 0x7f6cec03ad00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8006fd0 tx=0x7f6cf8006e40 comp rx=0 tx=0).stop 2023-07-14T11:53:18.950 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:18.966+0000 7f6d09b1d700 1 -- 172.21.15.17:0/2565524263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d040ff5e0 msgr2=0x7f6d04194750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:18.950 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:18.966+0000 7f6d09b1d700 1 --2- 172.21.15.17:0/2565524263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d040ff5e0 0x7f6d04194750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4004f40 tx=0x7f6cf40057c0 comp rx=0 tx=0).stop 2023-07-14T11:53:18.950 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:18.966+0000 7f6d09b1d700 1 -- 172.21.15.17:0/2565524263 shutdown_connections 2023-07-14T11:53:18.951 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:18.966+0000 7f6d09b1d700 1 --2- 172.21.15.17:0/2565524263 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f6cec038850 0x7f6cec03ad00 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:18.951 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:18.966+0000 7f6d09b1d700 1 --2- 172.21.15.17:0/2565524263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6d040ff5e0 0x7f6d04194750 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:18.951 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:18.966+0000 7f6d09b1d700 1 -- 172.21.15.17:0/2565524263 >> 172.21.15.17:0/2565524263 conn(0x7f6d040fab90 msgr2=0x7f6d040fb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:18.951 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:18.966+0000 7f6d09b1d700 1 -- 172.21.15.17:0/2565524263 shutdown_connections 2023-07-14T11:53:18.951 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:18.966+0000 7f6d09b1d700 1 -- 172.21.15.17:0/2565524263 wait complete. 2023-07-14T11:53:19.478 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-07-14T11:53:19.478 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-07-14T11:53:20.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:19 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:20.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:19 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:20.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:19 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi017", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:53:20.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:19 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:20.148 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:19 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:53:20.148 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:19 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:20.148 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:19 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/2565524263' entity='client.admin' 2023-07-14T11:53:20.148 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:19 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:53:20.148 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:19 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:20.203 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T11:53:21.824 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:21.839+0000 7fd5bb3cf700 1 -- 172.21.15.17:0/3435366205 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5b40ff5e0 msgr2=0x7fd5b40ff9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:21.824 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:21.839+0000 7fd5bb3cf700 1 --2- 172.21.15.17:0/3435366205 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5b40ff5e0 0x7fd5b40ff9f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0009910 tx=0x7fd5b0009c20 comp rx=0 tx=0).stop 2023-07-14T11:53:21.824 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:21.839+0000 7fd5bb3cf700 1 -- 172.21.15.17:0/3435366205 shutdown_connections 2023-07-14T11:53:21.825 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:21.839+0000 7fd5bb3cf700 1 --2- 172.21.15.17:0/3435366205 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5b40ff5e0 0x7fd5b40ff9f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:21.825 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:21.839+0000 7fd5bb3cf700 1 -- 172.21.15.17:0/3435366205 >> 172.21.15.17:0/3435366205 conn(0x7fd5b40fab90 msgr2=0x7fd5b40fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:21.825 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:21.840+0000 7fd5bb3cf700 1 -- 172.21.15.17:0/3435366205 shutdown_connections 2023-07-14T11:53:21.826 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:21.840+0000 7fd5bb3cf700 1 -- 172.21.15.17:0/3435366205 wait complete. 2023-07-14T11:53:21.827 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:21.841+0000 7fd5bb3cf700 1 Processor -- start 2023-07-14T11:53:21.827 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:21.841+0000 7fd5bb3cf700 1 -- start start 2023-07-14T11:53:21.827 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:21.841+0000 7fd5bb3cf700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5b40ff5e0 0x7fd5b4196940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:21.827 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:21.841+0000 7fd5bb3cf700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b4196e80 con 0x7fd5b40ff5e0 2023-07-14T11:53:21.827 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:21.841+0000 7fd5b916b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5b40ff5e0 0x7fd5b4196940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:21.828 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:21.841+0000 7fd5b916b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5b40ff5e0 0x7fd5b4196940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59624/0 (socket says 172.21.15.17:59624) 2023-07-14T11:53:21.828 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:21.841+0000 7fd5b916b700 1 -- 172.21.15.17:0/2356365986 learned_addr learned my addr 172.21.15.17:0/2356365986 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:53:21.828 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:21.842+0000 7fd5b916b700 1 -- 172.21.15.17:0/2356365986 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b00095f0 con 0x7fd5b40ff5e0 2023-07-14T11:53:21.828 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:21.842+0000 7fd5b916b700 1 --2- 172.21.15.17:0/2356365986 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5b40ff5e0 0x7fd5b4196940 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0004f40 tx=0x7fd5b00057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:21.828 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:21.842+0000 7fd5aa7fc700 1 -- 172.21.15.17:0/2356365986 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd5b001d070 con 0x7fd5b40ff5e0 2023-07-14T11:53:21.828 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:21.842+0000 7fd5aa7fc700 1 -- 172.21.15.17:0/2356365986 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5b000bc10 con 0x7fd5b40ff5e0 2023-07-14T11:53:21.829 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:21.842+0000 7fd5bb3cf700 1 -- 172.21.15.17:0/2356365986 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5b4197080 con 0x7fd5b40ff5e0 2023-07-14T11:53:21.829 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:21.842+0000 7fd5aa7fc700 1 -- 172.21.15.17:0/2356365986 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd5b0003a70 con 0x7fd5b40ff5e0 2023-07-14T11:53:21.829 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:21.842+0000 7fd5bb3cf700 1 -- 172.21.15.17:0/2356365986 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5b4197520 con 0x7fd5b40ff5e0 2023-07-14T11:53:21.829 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:21.843+0000 7fd5aa7fc700 1 -- 172.21.15.17:0/2356365986 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fd5b0003ce0 con 0x7fd5b40ff5e0 2023-07-14T11:53:21.829 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:21.843+0000 7fd5bb3cf700 1 -- 172.21.15.17:0/2356365986 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5b41905b0 con 0x7fd5b40ff5e0 2023-07-14T11:53:21.830 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:21.844+0000 7fd5aa7fc700 1 --2- 172.21.15.17:0/2356365986 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fd5a0038800 0x7fd5a003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:21.830 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:21.844+0000 7fd5aa7fc700 1 -- 172.21.15.17:0/2356365986 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd5b004c530 con 0x7fd5b40ff5e0 2023-07-14T11:53:21.830 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:21.844+0000 7fd5b896a700 1 --2- 172.21.15.17:0/2356365986 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fd5a0038800 0x7fd5a003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:21.830 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:21.845+0000 7fd5b896a700 1 --2- 172.21.15.17:0/2356365986 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fd5a0038800 0x7fd5a003acb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4006fd0 tx=0x7fd5a4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:21.832 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:21.849+0000 7fd5aa7fc700 1 -- 172.21.15.17:0/2356365986 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd5b002aba0 con 0x7fd5b40ff5e0 2023-07-14T11:53:22.045 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:22.062+0000 7fd5bb3cf700 1 -- 172.21.15.17:0/2356365986 --> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fd5b40609e0 con 0x7fd5a0038800 2023-07-14T11:53:22.049 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:22.065+0000 7fd5aa7fc700 1 -- 172.21.15.17:0/2356365986 <== mgr.14152 v2:172.21.15.17:6800/4237449858 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd5b40609e0 con 0x7fd5a0038800 2023-07-14T11:53:22.053 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:22.068+0000 7fd5bb3cf700 1 -- 172.21.15.17:0/2356365986 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fd5a0038800 msgr2=0x7fd5a003acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:22.053 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:22.068+0000 7fd5bb3cf700 1 --2- 172.21.15.17:0/2356365986 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fd5a0038800 0x7fd5a003acb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4006fd0 tx=0x7fd5a4006e40 comp rx=0 tx=0).stop 2023-07-14T11:53:22.053 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:22.068+0000 7fd5bb3cf700 1 -- 172.21.15.17:0/2356365986 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5b40ff5e0 msgr2=0x7fd5b4196940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:22.053 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:22.068+0000 7fd5bb3cf700 1 --2- 172.21.15.17:0/2356365986 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5b40ff5e0 0x7fd5b4196940 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0004f40 tx=0x7fd5b00057c0 comp rx=0 tx=0).stop 2023-07-14T11:53:22.053 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:22.068+0000 7fd5bb3cf700 1 -- 172.21.15.17:0/2356365986 shutdown_connections 2023-07-14T11:53:22.054 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:22.068+0000 7fd5bb3cf700 1 --2- 172.21.15.17:0/2356365986 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fd5a0038800 0x7fd5a003acb0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:22.054 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:22.068+0000 7fd5bb3cf700 1 --2- 172.21.15.17:0/2356365986 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5b40ff5e0 0x7fd5b4196940 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:22.054 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:22.068+0000 7fd5bb3cf700 1 -- 172.21.15.17:0/2356365986 >> 172.21.15.17:0/2356365986 conn(0x7fd5b40fab90 msgr2=0x7fd5b40fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:22.054 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:22.068+0000 7fd5bb3cf700 1 -- 172.21.15.17:0/2356365986 shutdown_connections 2023-07-14T11:53:22.054 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:22.068+0000 7fd5bb3cf700 1 -- 172.21.15.17:0/2356365986 wait complete. 2023-07-14T11:53:22.483 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi093 2023-07-14T11:53:22.484 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-07-14T11:53:22.484 DEBUG:teuthology.orchestra.run.smithi093:> dd of=/etc/ceph/ceph.conf 2023-07-14T11:53:22.504 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-07-14T11:53:22.505 DEBUG:teuthology.orchestra.run.smithi093:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-14T11:53:22.564 INFO:tasks.cephadm:Adding host smithi093 to orchestrator... 2023-07-14T11:53:22.565 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph orch host add smithi093 2023-07-14T11:53:23.305 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T11:53:23.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:23 smithi017 ceph-mon[126339]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:53:23.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:23 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:23.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:23 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:53:23.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:23 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:53:23.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:23 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:53:23.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:23 smithi017 ceph-mon[126339]: Updating smithi017:/etc/ceph/ceph.conf 2023-07-14T11:53:23.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:23 smithi017 ceph-mon[126339]: Updating smithi017:/var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T11:53:24.333 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:24 smithi017 ceph-mon[126339]: Updating smithi017:/etc/ceph/ceph.client.admin.keyring 2023-07-14T11:53:24.333 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:24 smithi017 ceph-mon[126339]: Updating smithi017:/var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T11:53:24.944 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:24.960+0000 7fdf198cd700 1 -- 172.21.15.17:0/3008909300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf14105630 msgr2=0x7fdf14105a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:24.945 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:24.960+0000 7fdf198cd700 1 --2- 172.21.15.17:0/3008909300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf14105630 0x7fdf14105a00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdf04009910 tx=0x7fdf04009c20 comp rx=0 tx=0).stop 2023-07-14T11:53:24.945 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:24.960+0000 7fdf198cd700 1 -- 172.21.15.17:0/3008909300 shutdown_connections 2023-07-14T11:53:24.945 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:24.960+0000 7fdf198cd700 1 --2- 172.21.15.17:0/3008909300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf14105630 0x7fdf14105a00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:24.945 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:24.960+0000 7fdf198cd700 1 -- 172.21.15.17:0/3008909300 >> 172.21.15.17:0/3008909300 conn(0x7fdf140fb130 msgr2=0x7fdf140fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:24.945 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:24.960+0000 7fdf198cd700 1 -- 172.21.15.17:0/3008909300 shutdown_connections 2023-07-14T11:53:24.945 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:24.960+0000 7fdf198cd700 1 -- 172.21.15.17:0/3008909300 wait complete. 2023-07-14T11:53:24.946 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:24.961+0000 7fdf198cd700 1 Processor -- start 2023-07-14T11:53:24.947 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:24.961+0000 7fdf198cd700 1 -- start start 2023-07-14T11:53:24.947 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:24.962+0000 7fdf198cd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf14105630 0x7fdf14072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:24.947 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:24.962+0000 7fdf198cd700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf14074790 con 0x7fdf14105630 2023-07-14T11:53:24.947 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:24.962+0000 7fdf12ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf14105630 0x7fdf14072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:24.947 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:24.962+0000 7fdf12ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf14105630 0x7fdf14072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58220/0 (socket says 172.21.15.17:58220) 2023-07-14T11:53:24.948 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:24.962+0000 7fdf12ffd700 1 -- 172.21.15.17:0/3836441298 learned_addr learned my addr 172.21.15.17:0/3836441298 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:53:24.948 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:24.962+0000 7fdf12ffd700 1 -- 172.21.15.17:0/3836441298 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf040095f0 con 0x7fdf14105630 2023-07-14T11:53:24.948 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:24.963+0000 7fdf12ffd700 1 --2- 172.21.15.17:0/3836441298 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf14105630 0x7fdf14072810 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdf04006010 tx=0x7fdf04004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:24.948 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:24.963+0000 7fdf188cb700 1 -- 172.21.15.17:0/3836441298 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf0401d070 con 0x7fdf14105630 2023-07-14T11:53:24.948 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:24.963+0000 7fdf188cb700 1 -- 172.21.15.17:0/3836441298 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf0400bc10 con 0x7fdf14105630 2023-07-14T11:53:24.948 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:24.963+0000 7fdf188cb700 1 -- 172.21.15.17:0/3836441298 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf04003910 con 0x7fdf14105630 2023-07-14T11:53:24.948 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:24.963+0000 7fdf198cd700 1 -- 172.21.15.17:0/3836441298 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf14072d50 con 0x7fdf14105630 2023-07-14T11:53:24.949 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:24.963+0000 7fdf198cd700 1 -- 172.21.15.17:0/3836441298 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf14073250 con 0x7fdf14105630 2023-07-14T11:53:24.949 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:24.964+0000 7fdf188cb700 1 -- 172.21.15.17:0/3836441298 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fdf0400bd80 con 0x7fdf14105630 2023-07-14T11:53:24.950 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:24.964+0000 7fdefa7fc700 1 -- 172.21.15.17:0/3836441298 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf14061b40 con 0x7fdf14105630 2023-07-14T11:53:24.950 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:24.965+0000 7fdf188cb700 1 --2- 172.21.15.17:0/3836441298 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fdefc038800 0x7fdefc03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:24.950 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:24.965+0000 7fdf188cb700 1 -- 172.21.15.17:0/3836441298 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fdf0404c4a0 con 0x7fdf14105630 2023-07-14T11:53:24.950 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:24.965+0000 7fdf127fc700 1 --2- 172.21.15.17:0/3836441298 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fdefc038800 0x7fdefc03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:24.950 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:24.966+0000 7fdf127fc700 1 --2- 172.21.15.17:0/3836441298 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fdefc038800 0x7fdefc03acb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdf08006fd0 tx=0x7fdf08006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:24.953 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:24.970+0000 7fdf188cb700 1 -- 172.21.15.17:0/3836441298 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fdf0402a7c0 con 0x7fdf14105630 2023-07-14T11:53:25.165 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:25.181+0000 7fdefa7fc700 1 -- 172.21.15.17:0/3836441298 --> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi093", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf14073550 con 0x7fdefc038800 2023-07-14T11:53:25.647 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:25 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:25.647 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:25 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:25.647 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:25 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:26.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:26 smithi017 ceph-mon[126339]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi093", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:53:26.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:26 smithi017 ceph-mon[126339]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:53:26.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:26 smithi017 ceph-mon[126339]: Deploying cephadm binary to smithi093 2023-07-14T11:53:27.537 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:27.553+0000 7fdf188cb700 1 -- 172.21.15.17:0/3836441298 <== mgr.14152 v2:172.21.15.17:6800/4237449858 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7fdf14073550 con 0x7fdefc038800 2023-07-14T11:53:27.537 INFO:teuthology.orchestra.run.smithi017.stdout:Added host 'smithi093' with addr '172.21.15.93' 2023-07-14T11:53:27.540 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:27.556+0000 7fdefa7fc700 1 -- 172.21.15.17:0/3836441298 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fdefc038800 msgr2=0x7fdefc03acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:27.541 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:27.556+0000 7fdefa7fc700 1 --2- 172.21.15.17:0/3836441298 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fdefc038800 0x7fdefc03acb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdf08006fd0 tx=0x7fdf08006e40 comp rx=0 tx=0).stop 2023-07-14T11:53:27.541 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:27.556+0000 7fdefa7fc700 1 -- 172.21.15.17:0/3836441298 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf14105630 msgr2=0x7fdf14072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:27.542 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:27.556+0000 7fdefa7fc700 1 --2- 172.21.15.17:0/3836441298 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf14105630 0x7fdf14072810 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdf04006010 tx=0x7fdf04004ff0 comp rx=0 tx=0).stop 2023-07-14T11:53:27.542 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:27.556+0000 7fdefa7fc700 1 -- 172.21.15.17:0/3836441298 shutdown_connections 2023-07-14T11:53:27.542 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:27.556+0000 7fdefa7fc700 1 --2- 172.21.15.17:0/3836441298 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fdefc038800 0x7fdefc03acb0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:27.543 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:27.556+0000 7fdefa7fc700 1 --2- 172.21.15.17:0/3836441298 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf14105630 0x7fdf14072810 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:27.543 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:27.556+0000 7fdefa7fc700 1 -- 172.21.15.17:0/3836441298 >> 172.21.15.17:0/3836441298 conn(0x7fdf140fb130 msgr2=0x7fdf140fbd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:27.543 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:27.556+0000 7fdefa7fc700 1 -- 172.21.15.17:0/3836441298 shutdown_connections 2023-07-14T11:53:27.544 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:27.557+0000 7fdefa7fc700 1 -- 172.21.15.17:0/3836441298 wait complete. 2023-07-14T11:53:27.999 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph orch host ls --format=json 2023-07-14T11:53:28.732 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T11:53:28.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:28 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:28.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:28 smithi017 ceph-mon[126339]: Added host smithi093 2023-07-14T11:53:28.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:28 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:53:28.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:28 smithi017 ceph-mon[126339]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:53:28.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:28 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:29.647 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:29 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:30.317 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:30.333+0000 7fb51abaa700 1 -- 172.21.15.17:0/50802309 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb514105630 msgr2=0x7fb514105a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:30.317 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:30.333+0000 7fb51abaa700 1 --2- 172.21.15.17:0/50802309 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb514105630 0x7fb514105a00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb508009910 tx=0x7fb508009c20 comp rx=0 tx=0).stop 2023-07-14T11:53:30.318 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:30.333+0000 7fb51abaa700 1 -- 172.21.15.17:0/50802309 shutdown_connections 2023-07-14T11:53:30.318 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:30.333+0000 7fb51abaa700 1 --2- 172.21.15.17:0/50802309 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb514105630 0x7fb514105a00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:30.318 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:30.333+0000 7fb51abaa700 1 -- 172.21.15.17:0/50802309 >> 172.21.15.17:0/50802309 conn(0x7fb5140fb130 msgr2=0x7fb5140fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:30.318 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:30.333+0000 7fb51abaa700 1 -- 172.21.15.17:0/50802309 shutdown_connections 2023-07-14T11:53:30.318 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:30.333+0000 7fb51abaa700 1 -- 172.21.15.17:0/50802309 wait complete. 2023-07-14T11:53:30.319 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:30.334+0000 7fb51abaa700 1 Processor -- start 2023-07-14T11:53:30.320 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:30.334+0000 7fb51abaa700 1 -- start start 2023-07-14T11:53:30.320 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:30.335+0000 7fb51abaa700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb514105630 0x7fb51419c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:30.320 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:30.335+0000 7fb51abaa700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51419ce70 con 0x7fb514105630 2023-07-14T11:53:30.320 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:30.335+0000 7fb518946700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb514105630 0x7fb51419c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:30.320 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:30.335+0000 7fb518946700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb514105630 0x7fb51419c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58234/0 (socket says 172.21.15.17:58234) 2023-07-14T11:53:30.320 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:30.335+0000 7fb518946700 1 -- 172.21.15.17:0/3563252133 learned_addr learned my addr 172.21.15.17:0/3563252133 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:53:30.320 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:30.335+0000 7fb518946700 1 -- 172.21.15.17:0/3563252133 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5080095f0 con 0x7fb514105630 2023-07-14T11:53:30.321 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:30.335+0000 7fb518946700 1 --2- 172.21.15.17:0/3563252133 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb514105630 0x7fb51419c930 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb508004f40 tx=0x7fb508005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:30.321 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:30.336+0000 7fb511ffb700 1 -- 172.21.15.17:0/3563252133 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb50801d070 con 0x7fb514105630 2023-07-14T11:53:30.321 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:30.336+0000 7fb51abaa700 1 -- 172.21.15.17:0/3563252133 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb51419d0d0 con 0x7fb514105630 2023-07-14T11:53:30.321 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:30.336+0000 7fb511ffb700 1 -- 172.21.15.17:0/3563252133 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb508004370 con 0x7fb514105630 2023-07-14T11:53:30.321 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:30.336+0000 7fb511ffb700 1 -- 172.21.15.17:0/3563252133 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb5080039b0 con 0x7fb514105630 2023-07-14T11:53:30.322 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:30.336+0000 7fb51abaa700 1 -- 172.21.15.17:0/3563252133 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5141969b0 con 0x7fb514105630 2023-07-14T11:53:30.322 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:30.337+0000 7fb511ffb700 1 -- 172.21.15.17:0/3563252133 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fb508003c70 con 0x7fb514105630 2023-07-14T11:53:30.322 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:30.337+0000 7fb51abaa700 1 -- 172.21.15.17:0/3563252133 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb514061b40 con 0x7fb514105630 2023-07-14T11:53:30.322 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:30.337+0000 7fb511ffb700 1 --2- 172.21.15.17:0/3563252133 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fb5040387b0 0x7fb50403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:30.323 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:30.338+0000 7fb511ffb700 1 -- 172.21.15.17:0/3563252133 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb50804c5e0 con 0x7fb514105630 2023-07-14T11:53:30.323 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:30.338+0000 7fb513fff700 1 --2- 172.21.15.17:0/3563252133 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fb5040387b0 0x7fb50403ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:30.323 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:30.338+0000 7fb513fff700 1 --2- 172.21.15.17:0/3563252133 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fb5040387b0 0x7fb50403ac60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb500006fd0 tx=0x7fb500006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:30.326 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:30.343+0000 7fb511ffb700 1 -- 172.21.15.17:0/3563252133 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb50802a980 con 0x7fb514105630 2023-07-14T11:53:30.536 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:30.553+0000 7fb51abaa700 1 -- 172.21.15.17:0/3563252133 --> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb514197470 con 0x7fb5040387b0 2023-07-14T11:53:30.538 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:30.554+0000 7fb511ffb700 1 -- 172.21.15.17:0/3563252133 <== mgr.14152 v2:172.21.15.17:6800/4237449858 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7fb514197470 con 0x7fb5040387b0 2023-07-14T11:53:30.538 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:53:30.538 INFO:teuthology.orchestra.run.smithi017.stdout:[{"addr": "172.21.15.17", "hostname": "smithi017", "labels": [], "status": ""}, {"addr": "172.21.15.93", "hostname": "smithi093", "labels": [], "status": ""}] 2023-07-14T11:53:30.540 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:30.556+0000 7fb51abaa700 1 -- 172.21.15.17:0/3563252133 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fb5040387b0 msgr2=0x7fb50403ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:30.540 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:30.556+0000 7fb51abaa700 1 --2- 172.21.15.17:0/3563252133 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fb5040387b0 0x7fb50403ac60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb500006fd0 tx=0x7fb500006e40 comp rx=0 tx=0).stop 2023-07-14T11:53:30.541 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:30.557+0000 7fb51abaa700 1 -- 172.21.15.17:0/3563252133 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb514105630 msgr2=0x7fb51419c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:30.541 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:30.557+0000 7fb51abaa700 1 --2- 172.21.15.17:0/3563252133 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb514105630 0x7fb51419c930 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb508004f40 tx=0x7fb508005020 comp rx=0 tx=0).stop 2023-07-14T11:53:30.541 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:30.557+0000 7fb51abaa700 1 -- 172.21.15.17:0/3563252133 shutdown_connections 2023-07-14T11:53:30.541 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:30.557+0000 7fb51abaa700 1 --2- 172.21.15.17:0/3563252133 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fb5040387b0 0x7fb50403ac60 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:30.541 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:30.557+0000 7fb51abaa700 1 --2- 172.21.15.17:0/3563252133 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb514105630 0x7fb51419c930 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:30.541 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:30.557+0000 7fb51abaa700 1 -- 172.21.15.17:0/3563252133 >> 172.21.15.17:0/3563252133 conn(0x7fb5140fb130 msgr2=0x7fb514074200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:30.541 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:30.557+0000 7fb51abaa700 1 -- 172.21.15.17:0/3563252133 shutdown_connections 2023-07-14T11:53:30.542 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:30.557+0000 7fb51abaa700 1 -- 172.21.15.17:0/3563252133 wait complete. 2023-07-14T11:53:31.107 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi097 2023-07-14T11:53:31.108 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-14T11:53:31.108 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/etc/ceph/ceph.conf 2023-07-14T11:53:31.129 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-14T11:53:31.129 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-14T11:53:31.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:30 smithi017 ceph-mon[126339]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:53:31.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:30 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:31.188 INFO:tasks.cephadm:Adding host smithi097 to orchestrator... 2023-07-14T11:53:31.188 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph orch host add smithi097 2023-07-14T11:53:31.912 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T11:53:32.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:31 smithi017 ceph-mon[126339]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T11:53:33.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:32 smithi017 ceph-mon[126339]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:53:33.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:32 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:33.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:32 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:33.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:32 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:33.148 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:32 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:33.148 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:32 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi093", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:53:33.148 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:32 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:53:33.148 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:32 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:53:33.546 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:33.562+0000 7feb7a6c6700 1 -- 172.21.15.17:0/4101199258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb740728b0 msgr2=0x7feb74072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:33.547 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:33.562+0000 7feb7a6c6700 1 --2- 172.21.15.17:0/4101199258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb740728b0 0x7feb74072c80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7feb64009910 tx=0x7feb64009c20 comp rx=0 tx=0).stop 2023-07-14T11:53:33.547 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:33.562+0000 7feb7a6c6700 1 -- 172.21.15.17:0/4101199258 shutdown_connections 2023-07-14T11:53:33.547 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:33.562+0000 7feb7a6c6700 1 --2- 172.21.15.17:0/4101199258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb740728b0 0x7feb74072c80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:33.547 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:33.562+0000 7feb7a6c6700 1 -- 172.21.15.17:0/4101199258 >> 172.21.15.17:0/4101199258 conn(0x7feb740fb130 msgr2=0x7feb740fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:33.547 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:33.562+0000 7feb7a6c6700 1 -- 172.21.15.17:0/4101199258 shutdown_connections 2023-07-14T11:53:33.547 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:33.563+0000 7feb7a6c6700 1 -- 172.21.15.17:0/4101199258 wait complete. 2023-07-14T11:53:33.548 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:33.563+0000 7feb7a6c6700 1 Processor -- start 2023-07-14T11:53:33.549 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:33.564+0000 7feb7a6c6700 1 -- start start 2023-07-14T11:53:33.549 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:33.564+0000 7feb7a6c6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb740728b0 0x7feb7419ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:33.549 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:33.564+0000 7feb7a6c6700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7419cf40 con 0x7feb740728b0 2023-07-14T11:53:33.549 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:33.564+0000 7feb73fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb740728b0 0x7feb7419ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:33.549 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:33.564+0000 7feb73fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb740728b0 0x7feb7419ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50058/0 (socket says 172.21.15.17:50058) 2023-07-14T11:53:33.549 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:33.564+0000 7feb73fff700 1 -- 172.21.15.17:0/3039682919 learned_addr learned my addr 172.21.15.17:0/3039682919 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:53:33.549 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:33.565+0000 7feb73fff700 1 -- 172.21.15.17:0/3039682919 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb640095f0 con 0x7feb740728b0 2023-07-14T11:53:33.550 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:33.565+0000 7feb73fff700 1 --2- 172.21.15.17:0/3039682919 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb740728b0 0x7feb7419ca00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7feb64005850 tx=0x7feb64004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:33.550 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:33.565+0000 7feb717fa700 1 -- 172.21.15.17:0/3039682919 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feb6401d070 con 0x7feb740728b0 2023-07-14T11:53:33.550 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:33.565+0000 7feb717fa700 1 -- 172.21.15.17:0/3039682919 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb64004370 con 0x7feb740728b0 2023-07-14T11:53:33.550 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:33.565+0000 7feb7a6c6700 1 -- 172.21.15.17:0/3039682919 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb7419d140 con 0x7feb740728b0 2023-07-14T11:53:33.550 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:33.565+0000 7feb717fa700 1 -- 172.21.15.17:0/3039682919 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feb64003ab0 con 0x7feb740728b0 2023-07-14T11:53:33.551 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:33.565+0000 7feb7a6c6700 1 -- 172.21.15.17:0/3039682919 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb7419d560 con 0x7feb740728b0 2023-07-14T11:53:33.551 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:33.566+0000 7feb7a6c6700 1 -- 172.21.15.17:0/3039682919 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb74061b40 con 0x7feb740728b0 2023-07-14T11:53:33.551 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:33.566+0000 7feb717fa700 1 -- 172.21.15.17:0/3039682919 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7feb64003d20 con 0x7feb740728b0 2023-07-14T11:53:33.552 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:33.567+0000 7feb717fa700 1 --2- 172.21.15.17:0/3039682919 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7feb5c038800 0x7feb5c03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:33.552 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:33.567+0000 7feb717fa700 1 -- 172.21.15.17:0/3039682919 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7feb6404c4f0 con 0x7feb740728b0 2023-07-14T11:53:33.552 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:33.567+0000 7feb737fe700 1 --2- 172.21.15.17:0/3039682919 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7feb5c038800 0x7feb5c03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:33.552 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:33.568+0000 7feb737fe700 1 --2- 172.21.15.17:0/3039682919 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7feb5c038800 0x7feb5c03acb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7feb68006fd0 tx=0x7feb68006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:33.555 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:33.572+0000 7feb717fa700 1 -- 172.21.15.17:0/3039682919 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7feb640044e0 con 0x7feb740728b0 2023-07-14T11:53:33.767 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:33.783+0000 7feb7a6c6700 1 -- 172.21.15.17:0/3039682919 --> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi097", "target": ["mon-mgr", ""]}) v1 -- 0x7feb74108c50 con 0x7feb5c038800 2023-07-14T11:53:34.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:33 smithi017 ceph-mon[126339]: Updating smithi093:/etc/ceph/ceph.conf 2023-07-14T11:53:34.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:33 smithi017 ceph-mon[126339]: Updating smithi093:/var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T11:53:34.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:33 smithi017 ceph-mon[126339]: Updating smithi093:/etc/ceph/ceph.client.admin.keyring 2023-07-14T11:53:34.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:33 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:34.148 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:33 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:34.148 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:33 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:35.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:34 smithi017 ceph-mon[126339]: Updating smithi093:/var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T11:53:35.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:34 smithi017 ceph-mon[126339]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:53:35.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:34 smithi017 ceph-mon[126339]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi097", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:53:36.130 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:36.146+0000 7feb717fa700 1 -- 172.21.15.17:0/3039682919 <== mgr.14152 v2:172.21.15.17:6800/4237449858 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7feb74108c50 con 0x7feb5c038800 2023-07-14T11:53:36.130 INFO:teuthology.orchestra.run.smithi017.stdout:Added host 'smithi097' with addr '172.21.15.97' 2023-07-14T11:53:36.132 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:36.149+0000 7feb7a6c6700 1 -- 172.21.15.17:0/3039682919 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7feb5c038800 msgr2=0x7feb5c03acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:36.132 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:36.149+0000 7feb7a6c6700 1 --2- 172.21.15.17:0/3039682919 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7feb5c038800 0x7feb5c03acb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7feb68006fd0 tx=0x7feb68006e40 comp rx=0 tx=0).stop 2023-07-14T11:53:36.133 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:36.149+0000 7feb7a6c6700 1 -- 172.21.15.17:0/3039682919 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb740728b0 msgr2=0x7feb7419ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:36.133 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:36.149+0000 7feb7a6c6700 1 --2- 172.21.15.17:0/3039682919 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb740728b0 0x7feb7419ca00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7feb64005850 tx=0x7feb64004ff0 comp rx=0 tx=0).stop 2023-07-14T11:53:36.133 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:36.149+0000 7feb7a6c6700 1 -- 172.21.15.17:0/3039682919 shutdown_connections 2023-07-14T11:53:36.133 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:36.149+0000 7feb7a6c6700 1 --2- 172.21.15.17:0/3039682919 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7feb5c038800 0x7feb5c03acb0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:36.133 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:36.149+0000 7feb7a6c6700 1 --2- 172.21.15.17:0/3039682919 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb740728b0 0x7feb7419ca00 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:36.133 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:36.149+0000 7feb7a6c6700 1 -- 172.21.15.17:0/3039682919 >> 172.21.15.17:0/3039682919 conn(0x7feb740fb130 msgr2=0x7feb74106130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:36.133 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:36.149+0000 7feb7a6c6700 1 -- 172.21.15.17:0/3039682919 shutdown_connections 2023-07-14T11:53:36.134 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:36.149+0000 7feb7a6c6700 1 -- 172.21.15.17:0/3039682919 wait complete. 2023-07-14T11:53:36.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:35 smithi017 ceph-mon[126339]: Deploying cephadm binary to smithi097 2023-07-14T11:53:36.607 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph orch host ls --format=json 2023-07-14T11:53:37.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:36 smithi017 ceph-mon[126339]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:53:37.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:36 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:37.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:36 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:53:37.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:36 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:37.345 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T11:53:38.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:37 smithi017 ceph-mon[126339]: Added host smithi097 2023-07-14T11:53:38.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:37 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:38.866 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:38.882+0000 7fd992ab4700 1 -- 172.21.15.17:0/4182897805 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd98c105650 msgr2=0x7fd98c105a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:38.867 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:38.882+0000 7fd992ab4700 1 --2- 172.21.15.17:0/4182897805 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd98c105650 0x7fd98c105a20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd980009910 tx=0x7fd980009c20 comp rx=0 tx=0).stop 2023-07-14T11:53:38.867 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:38.882+0000 7fd992ab4700 1 -- 172.21.15.17:0/4182897805 shutdown_connections 2023-07-14T11:53:38.867 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:38.882+0000 7fd992ab4700 1 --2- 172.21.15.17:0/4182897805 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd98c105650 0x7fd98c105a20 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:38.867 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:38.882+0000 7fd992ab4700 1 -- 172.21.15.17:0/4182897805 >> 172.21.15.17:0/4182897805 conn(0x7fd98c0fb130 msgr2=0x7fd98c0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:38.867 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:38.882+0000 7fd992ab4700 1 -- 172.21.15.17:0/4182897805 shutdown_connections 2023-07-14T11:53:38.867 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:38.883+0000 7fd992ab4700 1 -- 172.21.15.17:0/4182897805 wait complete. 2023-07-14T11:53:38.868 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:38.884+0000 7fd992ab4700 1 Processor -- start 2023-07-14T11:53:38.868 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:38.884+0000 7fd992ab4700 1 -- start start 2023-07-14T11:53:38.869 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:38.885+0000 7fd992ab4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd98c105650 0x7fd98c19a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:38.869 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:38.885+0000 7fd992ab4700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd98c19ac20 con 0x7fd98c105650 2023-07-14T11:53:38.869 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:38.885+0000 7fd990850700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd98c105650 0x7fd98c19a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:38.869 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:38.885+0000 7fd990850700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd98c105650 0x7fd98c19a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50078/0 (socket says 172.21.15.17:50078) 2023-07-14T11:53:38.869 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:38.885+0000 7fd990850700 1 -- 172.21.15.17:0/3459767292 learned_addr learned my addr 172.21.15.17:0/3459767292 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:53:38.870 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:38.885+0000 7fd990850700 1 -- 172.21.15.17:0/3459767292 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9800095f0 con 0x7fd98c105650 2023-07-14T11:53:38.870 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:38.886+0000 7fd990850700 1 --2- 172.21.15.17:0/3459767292 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd98c105650 0x7fd98c19a6e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd980006010 tx=0x7fd980004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:38.870 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:38.886+0000 7fd989ffb700 1 -- 172.21.15.17:0/3459767292 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd98001d070 con 0x7fd98c105650 2023-07-14T11:53:38.870 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:38.886+0000 7fd989ffb700 1 -- 172.21.15.17:0/3459767292 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd98000bc10 con 0x7fd98c105650 2023-07-14T11:53:38.870 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:38.886+0000 7fd992ab4700 1 -- 172.21.15.17:0/3459767292 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd98c19ae80 con 0x7fd98c105650 2023-07-14T11:53:38.871 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:38.886+0000 7fd989ffb700 1 -- 172.21.15.17:0/3459767292 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd980003910 con 0x7fd98c105650 2023-07-14T11:53:38.871 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:38.886+0000 7fd992ab4700 1 -- 172.21.15.17:0/3459767292 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd98c1947c0 con 0x7fd98c105650 2023-07-14T11:53:38.871 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:38.887+0000 7fd992ab4700 1 -- 172.21.15.17:0/3459767292 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd98c061b40 con 0x7fd98c105650 2023-07-14T11:53:38.872 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:38.887+0000 7fd989ffb700 1 -- 172.21.15.17:0/3459767292 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fd98000bd80 con 0x7fd98c105650 2023-07-14T11:53:38.872 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:38.888+0000 7fd989ffb700 1 --2- 172.21.15.17:0/3459767292 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fd97c038800 0x7fd97c03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:38.872 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:38.888+0000 7fd989ffb700 1 -- 172.21.15.17:0/3459767292 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd98004c560 con 0x7fd98c105650 2023-07-14T11:53:38.872 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:38.888+0000 7fd98bfff700 1 --2- 172.21.15.17:0/3459767292 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fd97c038800 0x7fd97c03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:38.873 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:38.889+0000 7fd98bfff700 1 --2- 172.21.15.17:0/3459767292 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fd97c038800 0x7fd97c03acb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd978006fd0 tx=0x7fd978006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:38.876 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:38.893+0000 7fd989ffb700 1 -- 172.21.15.17:0/3459767292 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd980029360 con 0x7fd98c105650 2023-07-14T11:53:39.088 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:39.105+0000 7fd992ab4700 1 -- 172.21.15.17:0/3459767292 --> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd98c194ac0 con 0x7fd97c038800 2023-07-14T11:53:39.089 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:39.106+0000 7fd989ffb700 1 -- 172.21.15.17:0/3459767292 <== mgr.14152 v2:172.21.15.17:6800/4237449858 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+238 (secure 0 0 0) 0x7fd98c194ac0 con 0x7fd97c038800 2023-07-14T11:53:39.089 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:53:39.090 INFO:teuthology.orchestra.run.smithi017.stdout:[{"addr": "172.21.15.17", "hostname": "smithi017", "labels": [], "status": ""}, {"addr": "172.21.15.93", "hostname": "smithi093", "labels": [], "status": ""}, {"addr": "172.21.15.97", "hostname": "smithi097", "labels": [], "status": ""}] 2023-07-14T11:53:39.092 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:39.108+0000 7fd992ab4700 1 -- 172.21.15.17:0/3459767292 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fd97c038800 msgr2=0x7fd97c03acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:39.092 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:39.108+0000 7fd992ab4700 1 --2- 172.21.15.17:0/3459767292 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fd97c038800 0x7fd97c03acb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd978006fd0 tx=0x7fd978006e40 comp rx=0 tx=0).stop 2023-07-14T11:53:39.092 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:39.108+0000 7fd992ab4700 1 -- 172.21.15.17:0/3459767292 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd98c105650 msgr2=0x7fd98c19a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:39.092 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:39.108+0000 7fd992ab4700 1 --2- 172.21.15.17:0/3459767292 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd98c105650 0x7fd98c19a6e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd980006010 tx=0x7fd980004ff0 comp rx=0 tx=0).stop 2023-07-14T11:53:39.092 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:39.108+0000 7fd992ab4700 1 -- 172.21.15.17:0/3459767292 shutdown_connections 2023-07-14T11:53:39.093 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:39.108+0000 7fd992ab4700 1 --2- 172.21.15.17:0/3459767292 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fd97c038800 0x7fd97c03acb0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:39.093 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:39.108+0000 7fd992ab4700 1 --2- 172.21.15.17:0/3459767292 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd98c105650 0x7fd98c19a6e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:39.093 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:39.108+0000 7fd992ab4700 1 -- 172.21.15.17:0/3459767292 >> 172.21.15.17:0/3459767292 conn(0x7fd98c0fb130 msgr2=0x7fd98c0fbd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:39.093 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:39.109+0000 7fd992ab4700 1 -- 172.21.15.17:0/3459767292 shutdown_connections 2023-07-14T11:53:39.093 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:39.109+0000 7fd992ab4700 1 -- 172.21.15.17:0/3459767292 wait complete. 2023-07-14T11:53:39.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:38 smithi017 ceph-mon[126339]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:53:39.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:38 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:39.623 INFO:tasks.cephadm:Setting crush tunables to default 2023-07-14T11:53:39.624 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph osd crush tunables default 2023-07-14T11:53:40.356 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T11:53:41.001 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:40 smithi017 ceph-mon[126339]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T11:53:41.002 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:40 smithi017 ceph-mon[126339]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:53:41.970 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:41.986+0000 7f1e93ff6700 1 -- 172.21.15.17:0/1272804471 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e8c0728b0 msgr2=0x7f1e8c072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:41.970 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:41.986+0000 7f1e93ff6700 1 --2- 172.21.15.17:0/1272804471 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e8c0728b0 0x7f1e8c072c80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1e88009910 tx=0x7f1e88009c20 comp rx=0 tx=0).stop 2023-07-14T11:53:41.970 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:41.986+0000 7f1e93ff6700 1 -- 172.21.15.17:0/1272804471 shutdown_connections 2023-07-14T11:53:41.970 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:41.986+0000 7f1e93ff6700 1 --2- 172.21.15.17:0/1272804471 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e8c0728b0 0x7f1e8c072c80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:41.971 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:41.986+0000 7f1e93ff6700 1 -- 172.21.15.17:0/1272804471 >> 172.21.15.17:0/1272804471 conn(0x7f1e8c0fb130 msgr2=0x7f1e8c0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:41.971 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:41.986+0000 7f1e93ff6700 1 -- 172.21.15.17:0/1272804471 shutdown_connections 2023-07-14T11:53:41.971 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:41.986+0000 7f1e93ff6700 1 -- 172.21.15.17:0/1272804471 wait complete. 2023-07-14T11:53:41.972 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:41.987+0000 7f1e93ff6700 1 Processor -- start 2023-07-14T11:53:41.972 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:41.987+0000 7f1e93ff6700 1 -- start start 2023-07-14T11:53:41.972 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:41.988+0000 7f1e93ff6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e8c0728b0 0x7f1e8c19b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:41.972 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:41.988+0000 7f1e93ff6700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8c19b670 con 0x7f1e8c0728b0 2023-07-14T11:53:41.972 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:41.988+0000 7f1e91d92700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e8c0728b0 0x7f1e8c19b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:41.973 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:41.988+0000 7f1e91d92700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e8c0728b0 0x7f1e8c19b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50104/0 (socket says 172.21.15.17:50104) 2023-07-14T11:53:41.973 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:41.988+0000 7f1e91d92700 1 -- 172.21.15.17:0/109858388 learned_addr learned my addr 172.21.15.17:0/109858388 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:53:41.973 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:41.988+0000 7f1e91d92700 1 -- 172.21.15.17:0/109858388 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e880095f0 con 0x7f1e8c0728b0 2023-07-14T11:53:41.973 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:41.989+0000 7f1e91d92700 1 --2- 172.21.15.17:0/109858388 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e8c0728b0 0x7f1e8c19b130 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1e88005850 tx=0x7f1e88004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:41.974 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:41.989+0000 7f1e82ffd700 1 -- 172.21.15.17:0/109858388 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1e8801d070 con 0x7f1e8c0728b0 2023-07-14T11:53:41.974 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:41.989+0000 7f1e82ffd700 1 -- 172.21.15.17:0/109858388 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e88004370 con 0x7f1e8c0728b0 2023-07-14T11:53:41.974 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:41.989+0000 7f1e93ff6700 1 -- 172.21.15.17:0/109858388 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e8c19b870 con 0x7f1e8c0728b0 2023-07-14T11:53:41.974 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:41.989+0000 7f1e82ffd700 1 -- 172.21.15.17:0/109858388 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1e88003ab0 con 0x7f1e8c0728b0 2023-07-14T11:53:41.974 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:41.989+0000 7f1e93ff6700 1 -- 172.21.15.17:0/109858388 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e8c19bc90 con 0x7f1e8c0728b0 2023-07-14T11:53:41.975 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:41.990+0000 7f1e82ffd700 1 -- 172.21.15.17:0/109858388 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f1e88003d20 con 0x7f1e8c0728b0 2023-07-14T11:53:41.975 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:41.990+0000 7f1e93ff6700 1 -- 172.21.15.17:0/109858388 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e8c061b40 con 0x7f1e8c0728b0 2023-07-14T11:53:41.975 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:41.991+0000 7f1e82ffd700 1 --2- 172.21.15.17:0/109858388 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f1e78038850 0x7f1e7803ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:41.976 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:41.991+0000 7f1e82ffd700 1 -- 172.21.15.17:0/109858388 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1e8804c800 con 0x7f1e8c0728b0 2023-07-14T11:53:41.976 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:41.991+0000 7f1e91591700 1 --2- 172.21.15.17:0/109858388 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f1e78038850 0x7f1e7803ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:41.976 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:41.992+0000 7f1e91591700 1 --2- 172.21.15.17:0/109858388 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f1e78038850 0x7f1e7803ad00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c006fd0 tx=0x7f1e7c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:41.979 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:41.996+0000 7f1e82ffd700 1 -- 172.21.15.17:0/109858388 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1e88029770 con 0x7f1e8c0728b0 2023-07-14T11:53:42.196 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:42.213+0000 7f1e93ff6700 1 -- 172.21.15.17:0/109858388 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f1e8c102510 con 0x7f1e8c0728b0 2023-07-14T11:53:42.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:42 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:42.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:42 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:42.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:42 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:42.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:42 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:42.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:42 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi097", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:53:42.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:42 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:53:42.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:42 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:53:42.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:42 smithi017 ceph-mon[126339]: Updating smithi097:/etc/ceph/ceph.conf 2023-07-14T11:53:42.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:42 smithi017 ceph-mon[126339]: Updating smithi097:/var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T11:53:42.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:42 smithi017 ceph-mon[126339]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2023-07-14T11:53:42.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:42 smithi017 ceph-mon[126339]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:53:42.399 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:42 smithi017 ceph-mon[126339]: Updating smithi097:/var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T11:53:42.399 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:42 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:42.399 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:42 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:42.399 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:42 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:43.082 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:43.098+0000 7f1e82ffd700 1 -- 172.21.15.17:0/109858388 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f1e88026020 con 0x7f1e8c0728b0 2023-07-14T11:53:43.085 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:43.101+0000 7f1e93ff6700 1 -- 172.21.15.17:0/109858388 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f1e78038850 msgr2=0x7f1e7803ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:43.085 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:43.101+0000 7f1e93ff6700 1 --2- 172.21.15.17:0/109858388 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f1e78038850 0x7f1e7803ad00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c006fd0 tx=0x7f1e7c006e40 comp rx=0 tx=0).stop 2023-07-14T11:53:43.085 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:43.101+0000 7f1e93ff6700 1 -- 172.21.15.17:0/109858388 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e8c0728b0 msgr2=0x7f1e8c19b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:43.085 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:43.101+0000 7f1e93ff6700 1 --2- 172.21.15.17:0/109858388 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e8c0728b0 0x7f1e8c19b130 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1e88005850 tx=0x7f1e88004ff0 comp rx=0 tx=0).stop 2023-07-14T11:53:43.086 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:43.102+0000 7f1e93ff6700 1 -- 172.21.15.17:0/109858388 shutdown_connections 2023-07-14T11:53:43.086 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:43.102+0000 7f1e93ff6700 1 --2- 172.21.15.17:0/109858388 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f1e78038850 0x7f1e7803ad00 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:43.086 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:43.102+0000 7f1e93ff6700 1 --2- 172.21.15.17:0/109858388 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e8c0728b0 0x7f1e8c19b130 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:43.086 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:43.102+0000 7f1e93ff6700 1 -- 172.21.15.17:0/109858388 >> 172.21.15.17:0/109858388 conn(0x7f1e8c0fb130 msgr2=0x7f1e8c0fbd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:43.087 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:43.102+0000 7f1e93ff6700 1 -- 172.21.15.17:0/109858388 shutdown_connections 2023-07-14T11:53:43.087 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:53:43.102+0000 7f1e93ff6700 1 -- 172.21.15.17:0/109858388 wait complete. 2023-07-14T11:53:43.087 INFO:teuthology.orchestra.run.smithi017.stderr:adjusted tunables profile to default 2023-07-14T11:53:43.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:43 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/109858388' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-07-14T11:53:43.542 INFO:tasks.cephadm:Adding mon.a on smithi017 2023-07-14T11:53:43.569 INFO:tasks.cephadm:Adding mon.b on smithi093 2023-07-14T11:53:43.570 INFO:tasks.cephadm:Adding mon.c on smithi097 2023-07-14T11:53:43.570 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph orch apply mon '3;smithi017:172.21.15.17=a;smithi093:172.21.15.93=b;smithi097:172.21.15.97=c' 2023-07-14T11:53:44.159 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T11:53:44.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:44 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/109858388' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-07-14T11:53:44.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:44 smithi017 ceph-mon[126339]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T11:53:44.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:44 smithi017 ceph-mon[126339]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:53:45.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:45.336+0000 7f7403119700 1 -- 172.21.15.97:0/2747682310 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73fc1044f0 msgr2=0x7f73fc1048c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:45.336+0000 7f7403119700 1 --2- 172.21.15.97:0/2747682310 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73fc1044f0 0x7f73fc1048c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f73f0009910 tx=0x7f73f0009c20 comp rx=0 tx=0).stop 2023-07-14T11:53:45.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:45.336+0000 7f7403119700 1 -- 172.21.15.97:0/2747682310 shutdown_connections 2023-07-14T11:53:45.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:45.336+0000 7f7403119700 1 --2- 172.21.15.97:0/2747682310 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73fc1044f0 0x7f73fc1048c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:45.336+0000 7f7403119700 1 -- 172.21.15.97:0/2747682310 >> 172.21.15.97:0/2747682310 conn(0x7f73fc0fa010 msgr2=0x7f73fc0fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:45.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:45.336+0000 7f7403119700 1 -- 172.21.15.97:0/2747682310 shutdown_connections 2023-07-14T11:53:45.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:45.336+0000 7f7403119700 1 -- 172.21.15.97:0/2747682310 wait complete. 2023-07-14T11:53:45.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:45.337+0000 7f7403119700 1 Processor -- start 2023-07-14T11:53:45.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:45.337+0000 7f7403119700 1 -- start start 2023-07-14T11:53:45.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:45.337+0000 7f7403119700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73fc1044f0 0x7f73fc1959d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:45.337+0000 7f7403119700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc195f10 con 0x7f73fc1044f0 2023-07-14T11:53:45.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:45.338+0000 7f7400eb5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73fc1044f0 0x7f73fc1959d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:45.338+0000 7f7400eb5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73fc1044f0 0x7f73fc1959d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.97:52050/0 (socket says 172.21.15.97:52050) 2023-07-14T11:53:45.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:45.338+0000 7f7400eb5700 1 -- 172.21.15.97:0/1724565391 learned_addr learned my addr 172.21.15.97:0/1724565391 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-14T11:53:45.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:45.338+0000 7f7400eb5700 1 -- 172.21.15.97:0/1724565391 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f00095f0 con 0x7f73fc1044f0 2023-07-14T11:53:45.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:45.338+0000 7f7400eb5700 1 --2- 172.21.15.97:0/1724565391 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73fc1044f0 0x7f73fc1959d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f73f0004f40 tx=0x7f73f0005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:45.338+0000 7f73f9ffb700 1 -- 172.21.15.97:0/1724565391 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f73f001d070 con 0x7f73fc1044f0 2023-07-14T11:53:45.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:45.338+0000 7f7403119700 1 -- 172.21.15.97:0/1724565391 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73fc196110 con 0x7f73fc1044f0 2023-07-14T11:53:45.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:45.338+0000 7f73f9ffb700 1 -- 172.21.15.97:0/1724565391 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73f0004370 con 0x7f73fc1044f0 2023-07-14T11:53:45.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:45.338+0000 7f73f9ffb700 1 -- 172.21.15.97:0/1724565391 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f73f00039b0 con 0x7f73fc1044f0 2023-07-14T11:53:45.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:45.339+0000 7f7403119700 1 -- 172.21.15.97:0/1724565391 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73fc196530 con 0x7f73fc1044f0 2023-07-14T11:53:45.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:45.339+0000 7f7403119700 1 -- 172.21.15.97:0/1724565391 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73fc1027a0 con 0x7f73fc1044f0 2023-07-14T11:53:45.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:45.340+0000 7f73f9ffb700 1 -- 172.21.15.97:0/1724565391 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f73f0003c70 con 0x7f73fc1044f0 2023-07-14T11:53:45.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:45.340+0000 7f73f9ffb700 1 --2- 172.21.15.97:0/1724565391 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f73ec0387b0 0x7f73ec03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:45.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:45.340+0000 7f73f9ffb700 1 -- 172.21.15.97:0/1724565391 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f73f004c5a0 con 0x7f73fc1044f0 2023-07-14T11:53:45.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:45.340+0000 7f73fbfff700 1 --2- 172.21.15.97:0/1724565391 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f73ec0387b0 0x7f73ec03ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:45.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:45.340+0000 7f73fbfff700 1 --2- 172.21.15.97:0/1724565391 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f73ec0387b0 0x7f73ec03ac60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f73e8006fd0 tx=0x7f73e8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:45.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:45.345+0000 7f73f9ffb700 1 -- 172.21.15.97:0/1724565391 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f73f002a980 con 0x7f73fc1044f0 2023-07-14T11:53:45.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:45.539+0000 7f7403119700 1 -- 172.21.15.97:0/1724565391 --> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi017:172.21.15.17=a;smithi093:172.21.15.93=b;smithi097:172.21.15.97=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f73fc068590 con 0x7f73ec0387b0 2023-07-14T11:53:45.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:45.547+0000 7f73f9ffb700 1 -- 172.21.15.97:0/1724565391 <== mgr.14152 v2:172.21.15.17:6800/4237449858 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f73fc068590 con 0x7f73ec0387b0 2023-07-14T11:53:45.530 INFO:teuthology.orchestra.run.smithi097.stdout:Scheduled mon update... 2023-07-14T11:53:45.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:45.549+0000 7f7403119700 1 -- 172.21.15.97:0/1724565391 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f73ec0387b0 msgr2=0x7f73ec03ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:45.549+0000 7f7403119700 1 --2- 172.21.15.97:0/1724565391 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f73ec0387b0 0x7f73ec03ac60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f73e8006fd0 tx=0x7f73e8006e40 comp rx=0 tx=0).stop 2023-07-14T11:53:45.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:45.549+0000 7f7403119700 1 -- 172.21.15.97:0/1724565391 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73fc1044f0 msgr2=0x7f73fc1959d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:45.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:45.549+0000 7f7403119700 1 --2- 172.21.15.97:0/1724565391 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73fc1044f0 0x7f73fc1959d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f73f0004f40 tx=0x7f73f0005020 comp rx=0 tx=0).stop 2023-07-14T11:53:45.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:45.549+0000 7f7403119700 1 -- 172.21.15.97:0/1724565391 shutdown_connections 2023-07-14T11:53:45.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:45.549+0000 7f7403119700 1 --2- 172.21.15.97:0/1724565391 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f73ec0387b0 0x7f73ec03ac60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:45.549+0000 7f7403119700 1 --2- 172.21.15.97:0/1724565391 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73fc1044f0 0x7f73fc1959d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:45.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:45.549+0000 7f7403119700 1 -- 172.21.15.97:0/1724565391 >> 172.21.15.97:0/1724565391 conn(0x7f73fc0fa010 msgr2=0x7f73fc106900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:45.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:45.549+0000 7f7403119700 1 -- 172.21.15.97:0/1724565391 shutdown_connections 2023-07-14T11:53:45.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:45.549+0000 7f7403119700 1 -- 172.21.15.97:0/1724565391 wait complete. 2023-07-14T11:53:46.012 DEBUG:teuthology.orchestra.run.smithi093:mon.b> sudo journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mon.b.service 2023-07-14T11:53:46.015 DEBUG:teuthology.orchestra.run.smithi097:mon.c> sudo journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mon.c.service 2023-07-14T11:53:46.017 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-07-14T11:53:46.017 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph mon dump -f json 2023-07-14T11:53:46.045 INFO:journalctl@ceph.mon.c.smithi097.stdout:-- Logs begin at Fri 2023-07-14 11:24:16 UTC. -- 2023-07-14T11:53:46.051 INFO:journalctl@ceph.mon.b.smithi093.stdout:-- Logs begin at Fri 2023-07-14 11:24:16 UTC. -- 2023-07-14T11:53:46.599 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T11:53:46.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:46 smithi017 ceph-mon[126339]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi017:172.21.15.17=a;smithi093:172.21.15.93=b;smithi097:172.21.15.97=c", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:53:46.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:46 smithi017 ceph-mon[126339]: Saving service mon spec with placement smithi017:172.21.15.17=a;smithi093:172.21.15.93=b;smithi097:172.21.15.97=c;count:3 2023-07-14T11:53:46.898 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:46 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:46.898 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:46 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:53:46.898 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:46 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:53:46.898 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:46 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:53:46.899 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:46 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:46.899 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:46 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T11:53:46.899 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:46 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:53:46.899 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:46 smithi017 ceph-mon[126339]: Deploying daemon mon.c on smithi097 2023-07-14T11:53:46.900 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:46 smithi017 ceph-mon[126339]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:53:48.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:48.676+0000 7fc2a6afa700 1 -- 172.21.15.97:0/956083975 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc2a01044f0 msgr2=0x7fc2a01048c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:48.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:48.676+0000 7fc2a6afa700 1 --2- 172.21.15.97:0/956083975 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc2a01044f0 0x7fc2a01048c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc288009910 tx=0x7fc288009c20 comp rx=0 tx=0).stop 2023-07-14T11:53:48.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:48.676+0000 7fc2a6afa700 1 -- 172.21.15.97:0/956083975 shutdown_connections 2023-07-14T11:53:48.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:48.676+0000 7fc2a6afa700 1 --2- 172.21.15.97:0/956083975 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc2a01044f0 0x7fc2a01048c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:48.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:48.676+0000 7fc2a6afa700 1 -- 172.21.15.97:0/956083975 >> 172.21.15.97:0/956083975 conn(0x7fc2a00742b0 msgr2=0x7fc2a00746b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:48.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:48.676+0000 7fc2a6afa700 1 -- 172.21.15.97:0/956083975 shutdown_connections 2023-07-14T11:53:48.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:48.676+0000 7fc2a6afa700 1 -- 172.21.15.97:0/956083975 wait complete. 2023-07-14T11:53:48.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:48.677+0000 7fc2a6afa700 1 Processor -- start 2023-07-14T11:53:48.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:48.677+0000 7fc2a6afa700 1 -- start start 2023-07-14T11:53:48.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:48.677+0000 7fc2a6afa700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc2a01044f0 0x7fc2a0195940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:48.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:48.677+0000 7fc2a6afa700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a0195e80 con 0x7fc2a01044f0 2023-07-14T11:53:48.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:48.678+0000 7fc2a4896700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc2a01044f0 0x7fc2a0195940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:48.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:48.678+0000 7fc2a4896700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc2a01044f0 0x7fc2a0195940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.97:52056/0 (socket says 172.21.15.97:52056) 2023-07-14T11:53:48.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:48.678+0000 7fc2a4896700 1 -- 172.21.15.97:0/1473486968 learned_addr learned my addr 172.21.15.97:0/1473486968 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-14T11:53:48.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:48.678+0000 7fc2a4896700 1 -- 172.21.15.97:0/1473486968 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2880095f0 con 0x7fc2a01044f0 2023-07-14T11:53:48.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:48.678+0000 7fc2a4896700 1 --2- 172.21.15.97:0/1473486968 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc2a01044f0 0x7fc2a0195940 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc288006010 tx=0x7fc288004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:48.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:48.678+0000 7fc29dffb700 1 -- 172.21.15.97:0/1473486968 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc28801d070 con 0x7fc2a01044f0 2023-07-14T11:53:48.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:48.679+0000 7fc29dffb700 1 -- 172.21.15.97:0/1473486968 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc28800bc10 con 0x7fc2a01044f0 2023-07-14T11:53:48.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:48.679+0000 7fc29dffb700 1 -- 172.21.15.97:0/1473486968 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc288003910 con 0x7fc2a01044f0 2023-07-14T11:53:48.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:48.680+0000 7fc2a6afa700 1 -- 172.21.15.97:0/1473486968 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2a0196080 con 0x7fc2a01044f0 2023-07-14T11:53:48.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:48.681+0000 7fc2a6afa700 1 -- 172.21.15.97:0/1473486968 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a01964a0 con 0x7fc2a01044f0 2023-07-14T11:53:48.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:48.684+0000 7fc2a6afa700 1 -- 172.21.15.97:0/1473486968 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc280005320 con 0x7fc2a01044f0 2023-07-14T11:53:48.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:48.686+0000 7fc29dffb700 1 -- 172.21.15.97:0/1473486968 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fc28800bd80 con 0x7fc2a01044f0 2023-07-14T11:53:48.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:48.687+0000 7fc29dffb700 1 --2- 172.21.15.97:0/1473486968 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fc28c038850 0x7fc28c03ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:48.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:48.687+0000 7fc29dffb700 1 -- 172.21.15.97:0/1473486968 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc28804c600 con 0x7fc2a01044f0 2023-07-14T11:53:48.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:48.687+0000 7fc29ffff700 1 --2- 172.21.15.97:0/1473486968 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fc28c038850 0x7fc28c03ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:48.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:48.688+0000 7fc29ffff700 1 --2- 172.21.15.97:0/1473486968 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fc28c038850 0x7fc28c03ad00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc290006fd0 tx=0x7fc290006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:48.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:48.692+0000 7fc29dffb700 1 -- 172.21.15.97:0/1473486968 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc288029360 con 0x7fc2a01044f0 2023-07-14T11:53:48.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:48.976+0000 7fc2a6afa700 1 -- 172.21.15.97:0/1473486968 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc2800059f0 con 0x7fc2a01044f0 2023-07-14T11:53:48.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:48.977+0000 7fc29dffb700 1 -- 172.21.15.97:0/1473486968 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+736 (secure 0 0 0) 0x7fc288026070 con 0x7fc2a01044f0 2023-07-14T11:53:48.961 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-14T11:53:48.961 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":1,"fsid":"b3c386cc-223c-11ee-9b33-001a4aab830c","modified":"2023-07-14T11:51:56.547088Z","created":"2023-07-14T11:51:56.547088Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-07-14T11:53:48.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:48.980+0000 7fc2a6afa700 1 -- 172.21.15.97:0/1473486968 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fc28c038850 msgr2=0x7fc28c03ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:48.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:48.980+0000 7fc2a6afa700 1 --2- 172.21.15.97:0/1473486968 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fc28c038850 0x7fc28c03ad00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc290006fd0 tx=0x7fc290006e40 comp rx=0 tx=0).stop 2023-07-14T11:53:48.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:48.980+0000 7fc2a6afa700 1 -- 172.21.15.97:0/1473486968 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc2a01044f0 msgr2=0x7fc2a0195940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:48.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:48.980+0000 7fc2a6afa700 1 --2- 172.21.15.97:0/1473486968 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc2a01044f0 0x7fc2a0195940 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc288006010 tx=0x7fc288004ff0 comp rx=0 tx=0).stop 2023-07-14T11:53:48.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:48.980+0000 7fc2a6afa700 1 -- 172.21.15.97:0/1473486968 shutdown_connections 2023-07-14T11:53:48.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:48.980+0000 7fc2a6afa700 1 --2- 172.21.15.97:0/1473486968 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fc28c038850 0x7fc28c03ad00 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:48.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:48.980+0000 7fc2a6afa700 1 --2- 172.21.15.97:0/1473486968 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc2a01044f0 0x7fc2a0195940 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:48.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:48.980+0000 7fc2a6afa700 1 -- 172.21.15.97:0/1473486968 >> 172.21.15.97:0/1473486968 conn(0x7fc2a00742b0 msgr2=0x7fc2a00fcce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:48.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:48.980+0000 7fc2a6afa700 1 -- 172.21.15.97:0/1473486968 shutdown_connections 2023-07-14T11:53:48.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:48.980+0000 7fc2a6afa700 1 -- 172.21.15.97:0/1473486968 wait complete. 2023-07-14T11:53:48.965 INFO:teuthology.orchestra.run.smithi097.stderr:dumped monmap epoch 1 2023-07-14T11:53:49.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:48 smithi017 ceph-mon[126339]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:53:49.628 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 podman[129271]: 2023-07-14T11:53:49.628 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: set uid:gid to 167:167 (ceph:ceph) 2023-07-14T11:53:49.628 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev), process ceph-mon, pid 7 2023-07-14T11:53:49.628 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: pidfile_write: ignore empty --pid-file 2023-07-14T11:53:49.888 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 bash[129271]: afacccf97ff54884f48f9b0bbbcb45c8686e3447c149dd863ec70f76a0d20e38 2023-07-14T11:53:49.888 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: load: jerasure load: lrc 2023-07-14T11:53:49.888 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: RocksDB version: 7.9.2 2023-07-14T11:53:49.888 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Git sha 0 2023-07-14T11:53:49.888 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Compile date 2023-07-13 18:29:45 2023-07-14T11:53:49.889 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: DB SUMMARY 2023-07-14T11:53:49.889 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: DB Session ID: C2WIM6KIKR8PGV6C0UO4 2023-07-14T11:53:49.889 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: CURRENT file: CURRENT 2023-07-14T11:53:49.889 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: IDENTITY file: IDENTITY 2023-07-14T11:53:49.889 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-07-14T11:53:49.890 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2023-07-14T11:53:49.890 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2023-07-14T11:53:49.890 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.error_if_exists: 0 2023-07-14T11:53:49.890 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.create_if_missing: 0 2023-07-14T11:53:49.891 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.paranoid_checks: 1 2023-07-14T11:53:49.891 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.flush_verify_memtable_count: 1 2023-07-14T11:53:49.891 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-07-14T11:53:49.891 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-07-14T11:53:49.892 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.env: 0x563df5b54720 2023-07-14T11:53:49.892 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.fs: PosixFileSystem 2023-07-14T11:53:49.892 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.info_log: 0x563df78d9280 2023-07-14T11:53:49.892 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.max_file_opening_threads: 16 2023-07-14T11:53:49.893 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.statistics: (nil) 2023-07-14T11:53:49.893 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.use_fsync: 0 2023-07-14T11:53:49.893 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.max_log_file_size: 0 2023-07-14T11:53:49.893 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-07-14T11:53:49.893 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.log_file_time_to_roll: 0 2023-07-14T11:53:49.893 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.keep_log_file_num: 1000 2023-07-14T11:53:49.893 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.recycle_log_file_num: 0 2023-07-14T11:53:49.894 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.allow_fallocate: 1 2023-07-14T11:53:49.894 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.allow_mmap_reads: 0 2023-07-14T11:53:49.894 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.allow_mmap_writes: 0 2023-07-14T11:53:49.894 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.use_direct_reads: 0 2023-07-14T11:53:49.895 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-07-14T11:53:49.895 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.create_missing_column_families: 0 2023-07-14T11:53:49.895 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.db_log_dir: 2023-07-14T11:53:49.895 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.wal_dir: 2023-07-14T11:53:49.895 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.table_cache_numshardbits: 6 2023-07-14T11:53:49.896 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.WAL_ttl_seconds: 0 2023-07-14T11:53:49.896 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.WAL_size_limit_MB: 0 2023-07-14T11:53:49.896 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-07-14T11:53:49.896 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-07-14T11:53:49.896 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.is_fd_close_on_exec: 1 2023-07-14T11:53:49.896 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.advise_random_on_open: 1 2023-07-14T11:53:49.897 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.db_write_buffer_size: 0 2023-07-14T11:53:49.897 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.write_buffer_manager: 0x563df77fa3c0 2023-07-14T11:53:49.897 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-07-14T11:53:49.897 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-07-14T11:53:49.897 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.use_adaptive_mutex: 0 2023-07-14T11:53:49.897 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.rate_limiter: (nil) 2023-07-14T11:53:49.897 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-07-14T11:53:49.897 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.wal_recovery_mode: 2 2023-07-14T11:53:49.898 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.enable_thread_tracking: 0 2023-07-14T11:53:49.898 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.enable_pipelined_write: 0 2023-07-14T11:53:49.898 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.unordered_write: 0 2023-07-14T11:53:49.898 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-07-14T11:53:49.898 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-07-14T11:53:49.898 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-07-14T11:53:49.898 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-07-14T11:53:49.898 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.row_cache: None 2023-07-14T11:53:49.899 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.wal_filter: None 2023-07-14T11:53:49.899 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-07-14T11:53:49.899 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.allow_ingest_behind: 0 2023-07-14T11:53:49.899 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.two_write_queues: 0 2023-07-14T11:53:49.899 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.manual_wal_flush: 0 2023-07-14T11:53:49.899 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.wal_compression: 0 2023-07-14T11:53:49.899 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.atomic_flush: 0 2023-07-14T11:53:49.900 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-07-14T11:53:49.900 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.persist_stats_to_disk: 0 2023-07-14T11:53:49.900 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.write_dbid_to_manifest: 0 2023-07-14T11:53:49.900 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.log_readahead_size: 0 2023-07-14T11:53:49.900 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-07-14T11:53:49.900 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.best_efforts_recovery: 0 2023-07-14T11:53:49.900 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-07-14T11:53:49.901 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-07-14T11:53:49.901 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.allow_data_in_errors: 0 2023-07-14T11:53:49.901 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.db_host_id: __hostname__ 2023-07-14T11:53:49.901 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.enforce_single_del_contracts: true 2023-07-14T11:53:49.901 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.max_background_jobs: 2 2023-07-14T11:53:49.903 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.max_background_compactions: -1 2023-07-14T11:53:49.903 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.max_subcompactions: 1 2023-07-14T11:53:49.903 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-07-14T11:53:49.904 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-07-14T11:53:49.904 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.delayed_write_rate : 16777216 2023-07-14T11:53:49.904 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.max_total_wal_size: 0 2023-07-14T11:53:49.904 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-07-14T11:53:49.904 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.stats_dump_period_sec: 600 2023-07-14T11:53:49.905 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.stats_persist_period_sec: 600 2023-07-14T11:53:49.905 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-07-14T11:53:49.905 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.max_open_files: -1 2023-07-14T11:53:49.905 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.bytes_per_sync: 0 2023-07-14T11:53:49.905 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.wal_bytes_per_sync: 0 2023-07-14T11:53:49.905 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.strict_bytes_per_sync: 0 2023-07-14T11:53:49.906 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.compaction_readahead_size: 0 2023-07-14T11:53:49.906 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.max_background_flushes: -1 2023-07-14T11:53:49.906 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Compression algorithms supported: 2023-07-14T11:53:49.906 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-07-14T11:53:49.906 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: kZSTD supported: 0 2023-07-14T11:53:49.907 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: kXpressCompression supported: 0 2023-07-14T11:53:49.907 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: kLZ4HCCompression supported: 1 2023-07-14T11:53:49.907 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: kZlibCompression supported: 1 2023-07-14T11:53:49.907 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: kSnappyCompression supported: 1 2023-07-14T11:53:49.907 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: kLZ4Compression supported: 1 2023-07-14T11:53:49.908 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: kBZip2Compression supported: 0 2023-07-14T11:53:49.908 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Fast CRC32 supported: Supported on x86 2023-07-14T11:53:49.908 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: DMutex implementation: pthread_mutex_t 2023-07-14T11:53:49.908 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2023-07-14T11:53:49.908 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-07-14T11:53:49.908 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-07-14T11:53:49.908 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.merge_operator: 2023-07-14T11:53:49.909 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.compaction_filter: None 2023-07-14T11:53:49.909 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.compaction_filter_factory: None 2023-07-14T11:53:49.909 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.sst_partitioner_factory: None 2023-07-14T11:53:49.909 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.memtable_factory: SkipListFactory 2023-07-14T11:53:49.909 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.table_factory: BlockBasedTable 2023-07-14T11:53:49.909 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x563df78d9380) 2023-07-14T11:53:49.909 INFO:journalctl@ceph.mon.c.smithi097.stdout: cache_index_and_filter_blocks: 1 2023-07-14T11:53:49.909 INFO:journalctl@ceph.mon.c.smithi097.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-07-14T11:53:49.910 INFO:journalctl@ceph.mon.c.smithi097.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-07-14T11:53:49.910 INFO:journalctl@ceph.mon.c.smithi097.stdout: pin_top_level_index_and_filter: 1 2023-07-14T11:53:49.910 INFO:journalctl@ceph.mon.c.smithi097.stdout: index_type: 0 2023-07-14T11:53:49.910 INFO:journalctl@ceph.mon.c.smithi097.stdout: data_block_index_type: 0 2023-07-14T11:53:49.910 INFO:journalctl@ceph.mon.c.smithi097.stdout: index_shortening: 1 2023-07-14T11:53:49.911 INFO:journalctl@ceph.mon.c.smithi097.stdout: data_block_hash_table_util_ratio: 0.750000 2023-07-14T11:53:49.911 INFO:journalctl@ceph.mon.c.smithi097.stdout: checksum: 4 2023-07-14T11:53:49.911 INFO:journalctl@ceph.mon.c.smithi097.stdout: no_block_cache: 0 2023-07-14T11:53:49.911 INFO:journalctl@ceph.mon.c.smithi097.stdout: block_cache: 0x563df78991f0 2023-07-14T11:53:49.911 INFO:journalctl@ceph.mon.c.smithi097.stdout: block_cache_name: BinnedLRUCache 2023-07-14T11:53:49.911 INFO:journalctl@ceph.mon.c.smithi097.stdout: block_cache_options: 2023-07-14T11:53:49.911 INFO:journalctl@ceph.mon.c.smithi097.stdout: capacity : 536870912 2023-07-14T11:53:49.911 INFO:journalctl@ceph.mon.c.smithi097.stdout: num_shard_bits : 4 2023-07-14T11:53:49.912 INFO:journalctl@ceph.mon.c.smithi097.stdout: strict_capacity_limit : 0 2023-07-14T11:53:49.912 INFO:journalctl@ceph.mon.c.smithi097.stdout: high_pri_pool_ratio: 0.000 2023-07-14T11:53:49.912 INFO:journalctl@ceph.mon.c.smithi097.stdout: block_cache_compressed: (nil) 2023-07-14T11:53:49.912 INFO:journalctl@ceph.mon.c.smithi097.stdout: persistent_cache: (nil) 2023-07-14T11:53:49.912 INFO:journalctl@ceph.mon.c.smithi097.stdout: block_size: 4096 2023-07-14T11:53:49.912 INFO:journalctl@ceph.mon.c.smithi097.stdout: block_size_deviation: 10 2023-07-14T11:53:49.912 INFO:journalctl@ceph.mon.c.smithi097.stdout: block_restart_interval: 16 2023-07-14T11:53:49.912 INFO:journalctl@ceph.mon.c.smithi097.stdout: index_block_restart_interval: 1 2023-07-14T11:53:49.913 INFO:journalctl@ceph.mon.c.smithi097.stdout: metadata_block_size: 4096 2023-07-14T11:53:49.913 INFO:journalctl@ceph.mon.c.smithi097.stdout: partition_filters: 0 2023-07-14T11:53:49.913 INFO:journalctl@ceph.mon.c.smithi097.stdout: use_delta_encoding: 1 2023-07-14T11:53:49.913 INFO:journalctl@ceph.mon.c.smithi097.stdout: filter_policy: bloomfilter 2023-07-14T11:53:49.913 INFO:journalctl@ceph.mon.c.smithi097.stdout: whole_key_filtering: 1 2023-07-14T11:53:49.913 INFO:journalctl@ceph.mon.c.smithi097.stdout: verify_compression: 0 2023-07-14T11:53:49.914 INFO:journalctl@ceph.mon.c.smithi097.stdout: read_amp_bytes_per_bit: 0 2023-07-14T11:53:49.914 INFO:journalctl@ceph.mon.c.smithi097.stdout: format_version: 5 2023-07-14T11:53:49.914 INFO:journalctl@ceph.mon.c.smithi097.stdout: enable_index_compression: 1 2023-07-14T11:53:49.914 INFO:journalctl@ceph.mon.c.smithi097.stdout: block_align: 0 2023-07-14T11:53:49.914 INFO:journalctl@ceph.mon.c.smithi097.stdout: max_auto_readahead_size: 262144 2023-07-14T11:53:49.914 INFO:journalctl@ceph.mon.c.smithi097.stdout: prepopulate_block_cache: 0 2023-07-14T11:53:49.914 INFO:journalctl@ceph.mon.c.smithi097.stdout: initial_auto_readahead_size: 8192 2023-07-14T11:53:49.915 INFO:journalctl@ceph.mon.c.smithi097.stdout: num_file_reads_for_auto_readahead: 2 2023-07-14T11:53:49.915 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.write_buffer_size: 33554432 2023-07-14T11:53:49.915 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.max_write_buffer_number: 2 2023-07-14T11:53:49.915 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.compression: NoCompression 2023-07-14T11:53:49.915 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.bottommost_compression: Disabled 2023-07-14T11:53:49.915 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.prefix_extractor: nullptr 2023-07-14T11:53:49.916 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-07-14T11:53:49.916 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.num_levels: 7 2023-07-14T11:53:49.916 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-07-14T11:53:49.916 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-07-14T11:53:49.916 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-07-14T11:53:49.916 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-07-14T11:53:49.917 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-07-14T11:53:49.917 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-07-14T11:53:49.917 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-07-14T11:53:49.917 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-07-14T11:53:49.917 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-07-14T11:53:49.917 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-07-14T11:53:49.918 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-07-14T11:53:49.918 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-07-14T11:53:49.918 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.compression_opts.window_bits: -14 2023-07-14T11:53:49.918 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.compression_opts.level: 32767 2023-07-14T11:53:49.918 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.compression_opts.strategy: 0 2023-07-14T11:53:49.918 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-07-14T11:53:49.918 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-07-14T11:53:49.919 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-07-14T11:53:49.919 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-07-14T11:53:49.919 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.compression_opts.enabled: false 2023-07-14T11:53:49.919 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-07-14T11:53:49.919 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-07-14T11:53:49.919 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-07-14T11:53:49.919 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-07-14T11:53:49.919 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.target_file_size_base: 67108864 2023-07-14T11:53:49.920 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.target_file_size_multiplier: 1 2023-07-14T11:53:49.920 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-07-14T11:53:49.920 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-07-14T11:53:49.920 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-07-14T11:53:49.920 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-07-14T11:53:49.920 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-07-14T11:53:49.920 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-07-14T11:53:49.921 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-07-14T11:53:49.921 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-07-14T11:53:49.921 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-07-14T11:53:49.921 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-07-14T11:53:49.921 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-07-14T11:53:49.922 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-07-14T11:53:49.922 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-07-14T11:53:49.922 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.arena_block_size: 1048576 2023-07-14T11:53:49.922 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-07-14T11:53:49.923 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-07-14T11:53:49.923 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.disable_auto_compactions: 0 2023-07-14T11:53:49.923 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-07-14T11:53:49.923 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-07-14T11:53:49.924 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-07-14T11:53:49.924 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-07-14T11:53:49.924 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-07-14T11:53:49.924 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-07-14T11:53:49.924 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-07-14T11:53:49.925 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-07-14T11:53:49.925 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-07-14T11:53:49.925 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-07-14T11:53:49.925 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.table_properties_collectors: 2023-07-14T11:53:49.925 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.inplace_update_support: 0 2023-07-14T11:53:49.925 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.inplace_update_num_locks: 10000 2023-07-14T11:53:49.926 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-07-14T11:53:49.926 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-07-14T11:53:49.926 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.memtable_huge_page_size: 0 2023-07-14T11:53:49.926 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.bloom_locality: 0 2023-07-14T11:53:49.926 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.max_successive_merges: 0 2023-07-14T11:53:49.927 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.optimize_filters_for_hits: 0 2023-07-14T11:53:49.927 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.paranoid_file_checks: 0 2023-07-14T11:53:49.927 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.force_consistency_checks: 1 2023-07-14T11:53:49.927 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.report_bg_io_stats: 0 2023-07-14T11:53:49.927 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.ttl: 2592000 2023-07-14T11:53:49.927 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.periodic_compaction_seconds: 0 2023-07-14T11:53:49.928 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-07-14T11:53:49.928 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-07-14T11:53:49.928 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.enable_blob_files: false 2023-07-14T11:53:49.928 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.min_blob_size: 0 2023-07-14T11:53:49.928 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.blob_file_size: 268435456 2023-07-14T11:53:49.928 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.blob_compression_type: NoCompression 2023-07-14T11:53:49.928 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.enable_blob_garbage_collection: false 2023-07-14T11:53:49.929 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-07-14T11:53:49.929 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-07-14T11:53:49.929 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-07-14T11:53:49.929 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.blob_file_starting_level: 0 2023-07-14T11:53:49.929 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-07-14T11:53:49.929 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-07-14T11:53:49.929 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-07-14T11:53:49.930 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 90c95dec-bf3e-4e46-94da-0e07e8fc29b4 2023-07-14T11:53:49.930 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689335629651362, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-07-14T11:53:49.930 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-07-14T11:53:49.930 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689335629651766, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1689335629, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "90c95dec-bf3e-4e46-94da-0e07e8fc29b4", "db_session_id": "C2WIM6KIKR8PGV6C0UO4", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-07-14T11:53:49.930 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689335629651832, "job": 1, "event": "recovery_finished"} 2023-07-14T11:53:49.930 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-07-14T11:53:49.931 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-07-14T11:53:49.931 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x563df7932000 2023-07-14T11:53:49.931 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: DB pointer 0x563df791e000 2023-07-14T11:53:49.931 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-07-14T11:53:49.931 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-07-14T11:53:49.931 INFO:journalctl@ceph.mon.c.smithi097.stdout: ** DB Stats ** 2023-07-14T11:53:49.931 INFO:journalctl@ceph.mon.c.smithi097.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-14T11:53:49.931 INFO:journalctl@ceph.mon.c.smithi097.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-07-14T11:53:49.932 INFO:journalctl@ceph.mon.c.smithi097.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-14T11:53:49.932 INFO:journalctl@ceph.mon.c.smithi097.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-14T11:53:49.932 INFO:journalctl@ceph.mon.c.smithi097.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-07-14T11:53:49.932 INFO:journalctl@ceph.mon.c.smithi097.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-14T11:53:49.932 INFO:journalctl@ceph.mon.c.smithi097.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-14T11:53:49.932 INFO:journalctl@ceph.mon.c.smithi097.stdout: 2023-07-14T11:53:49.932 INFO:journalctl@ceph.mon.c.smithi097.stdout: ** Compaction Stats [default] ** 2023-07-14T11:53:49.933 INFO:journalctl@ceph.mon.c.smithi097.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-14T11:53:49.933 INFO:journalctl@ceph.mon.c.smithi097.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-07-14T11:53:49.933 INFO:journalctl@ceph.mon.c.smithi097.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-14T11:53:49.933 INFO:journalctl@ceph.mon.c.smithi097.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-14T11:53:49.933 INFO:journalctl@ceph.mon.c.smithi097.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-14T11:53:49.933 INFO:journalctl@ceph.mon.c.smithi097.stdout: 2023-07-14T11:53:49.933 INFO:journalctl@ceph.mon.c.smithi097.stdout: ** Compaction Stats [default] ** 2023-07-14T11:53:49.933 INFO:journalctl@ceph.mon.c.smithi097.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-14T11:53:49.934 INFO:journalctl@ceph.mon.c.smithi097.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-07-14T11:53:49.934 INFO:journalctl@ceph.mon.c.smithi097.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-14T11:53:49.934 INFO:journalctl@ceph.mon.c.smithi097.stdout: 2023-07-14T11:53:49.934 INFO:journalctl@ceph.mon.c.smithi097.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-07-14T11:53:49.934 INFO:journalctl@ceph.mon.c.smithi097.stdout: 2023-07-14T11:53:49.934 INFO:journalctl@ceph.mon.c.smithi097.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-14T11:53:49.934 INFO:journalctl@ceph.mon.c.smithi097.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-07-14T11:53:49.934 INFO:journalctl@ceph.mon.c.smithi097.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-07-14T11:53:49.935 INFO:journalctl@ceph.mon.c.smithi097.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-07-14T11:53:49.935 INFO:journalctl@ceph.mon.c.smithi097.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-07-14T11:53:49.935 INFO:journalctl@ceph.mon.c.smithi097.stdout: AddFile(Keys): cumulative 0, interval 0 2023-07-14T11:53:49.935 INFO:journalctl@ceph.mon.c.smithi097.stdout: Cumulative compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-14T11:53:49.935 INFO:journalctl@ceph.mon.c.smithi097.stdout: Interval compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-14T11:53:49.935 INFO:journalctl@ceph.mon.c.smithi097.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-07-14T11:53:49.935 INFO:journalctl@ceph.mon.c.smithi097.stdout: Block cache BinnedLRUCache@0x563df78991f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2023-07-14T11:53:49.936 INFO:journalctl@ceph.mon.c.smithi097.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-07-14T11:53:49.936 INFO:journalctl@ceph.mon.c.smithi097.stdout: 2023-07-14T11:53:49.936 INFO:journalctl@ceph.mon.c.smithi097.stdout: ** File Read Latency Histogram By Level [default] ** 2023-07-14T11:53:49.936 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: mon.c does not exist in monmap, will attempt to join an existing cluster 2023-07-14T11:53:49.936 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: using public_addr v2:172.21.15.97:0/0 -> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] 2023-07-14T11:53:49.936 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: starting mon.c rank -1 at public addrs [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] at bind addrs [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid b3c386cc-223c-11ee-9b33-001a4aab830c 2023-07-14T11:53:49.937 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: mon.c@-1(???) e0 preinit fsid b3c386cc-223c-11ee-9b33-001a4aab830c 2023-07-14T11:53:49.937 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: mon.c@-1(synchronizing).mds e1 new map 2023-07-14T11:53:49.937 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: mon.c@-1(synchronizing).mds e1 print_map 2023-07-14T11:53:49.937 INFO:journalctl@ceph.mon.c.smithi097.stdout: e1 2023-07-14T11:53:49.937 INFO:journalctl@ceph.mon.c.smithi097.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-07-14T11:53:49.938 INFO:journalctl@ceph.mon.c.smithi097.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-14T11:53:49.938 INFO:journalctl@ceph.mon.c.smithi097.stdout: legacy client fscid: -1 2023-07-14T11:53:49.938 INFO:journalctl@ceph.mon.c.smithi097.stdout: 2023-07-14T11:53:49.938 INFO:journalctl@ceph.mon.c.smithi097.stdout: No filesystems configured 2023-07-14T11:53:49.938 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-07-14T11:53:49.938 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-07-14T11:53:49.938 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-07-14T11:53:49.938 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-07-14T11:53:49.939 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-07-14T11:53:49.939 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-07-14T11:53:49.939 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-07-14T11:53:49.939 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-14T11:53:49.939 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-14T11:53:49.939 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-14T11:53:49.939 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: Updating smithi093:/etc/ceph/ceph.conf 2023-07-14T11:53:49.940 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: Updating smithi093:/var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T11:53:49.940 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: Updating smithi093:/etc/ceph/ceph.client.admin.keyring 2023-07-14T11:53:49.940 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:49.940 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:49.940 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:49.940 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: Updating smithi093:/var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T11:53:49.940 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:53:49.940 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi097", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:53:49.941 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: Deploying cephadm binary to smithi097 2023-07-14T11:53:49.941 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:53:49.941 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:49.941 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:53:49.941 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:49.941 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: Added host smithi097 2023-07-14T11:53:49.941 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:49.942 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:53:49.942 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:49.942 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T11:53:49.943 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:53:49.943 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:49.943 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:49.943 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:49.944 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:49.944 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi097", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:53:49.944 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:53:49.944 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:53:49.945 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: Updating smithi097:/etc/ceph/ceph.conf 2023-07-14T11:53:49.945 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: Updating smithi097:/var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T11:53:49.945 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2023-07-14T11:53:49.945 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:53:49.945 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: Updating smithi097:/var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T11:53:49.946 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:49.946 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:49.946 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:49.946 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/109858388' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-07-14T11:53:49.946 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/109858388' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-07-14T11:53:49.946 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T11:53:49.947 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:53:49.947 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi017:172.21.15.17=a;smithi093:172.21.15.93=b;smithi097:172.21.15.97=c", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:53:49.947 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: Saving service mon spec with placement smithi017:172.21.15.17=a;smithi093:172.21.15.93=b;smithi097:172.21.15.97=c;count:3 2023-07-14T11:53:49.947 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:49.947 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:53:49.948 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:53:49.948 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:53:49.948 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:49.948 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T11:53:49.948 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:53:49.948 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: Deploying daemon mon.c on smithi097 2023-07-14T11:53:49.949 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:53:49.949 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:53:49.949 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-07-14T11:53:49.949 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 ceph-mon[129313]: mon.c@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete 2023-07-14T11:53:49.949 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:49 smithi097 systemd[1]: Started Ceph mon.c for b3c386cc-223c-11ee-9b33-001a4aab830c. 2023-07-14T11:53:50.458 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-07-14T11:53:50.459 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph mon dump -f json 2023-07-14T11:53:51.016 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T11:53:55.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:54 smithi097 ceph-mon[129313]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:53:55.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:54 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T11:53:55.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:54 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:53:55.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:54 smithi097 ceph-mon[129313]: mon.a calling monitor election 2023-07-14T11:53:55.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:54 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:53:55.139 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:54 smithi097 ceph-mon[129313]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:53:55.139 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:54 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:53:55.139 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:54 smithi097 ceph-mon[129313]: mon.c calling monitor election 2023-07-14T11:53:55.139 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:54 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:53:55.139 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:54 smithi097 ceph-mon[129313]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:53:55.139 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:54 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:53:55.139 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:54 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:53:55.140 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:54 smithi097 ceph-mon[129313]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-14T11:53:55.140 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:54 smithi097 ceph-mon[129313]: monmap e2: 2 mons at {a=[v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0],c=[v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0]} removed_ranks: {} 2023-07-14T11:53:55.140 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:54 smithi097 ceph-mon[129313]: fsmap 2023-07-14T11:53:55.140 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:54 smithi097 ceph-mon[129313]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T11:53:55.140 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:54 smithi097 ceph-mon[129313]: mgrmap e13: x(active, since 49s) 2023-07-14T11:53:55.140 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:54 smithi097 ceph-mon[129313]: overall HEALTH_OK 2023-07-14T11:53:55.140 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:54 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:55.141 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:54 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:55.141 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:54 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:55.141 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:54 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T11:53:55.141 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:54 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:53:55.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:54 smithi017 ceph-mon[126339]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:53:55.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:54 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T11:53:55.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:54 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:53:55.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:54 smithi017 ceph-mon[126339]: mon.a calling monitor election 2023-07-14T11:53:55.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:54 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:53:55.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:54 smithi017 ceph-mon[126339]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:53:55.148 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:54 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:53:55.148 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:54 smithi017 ceph-mon[126339]: mon.c calling monitor election 2023-07-14T11:53:55.148 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:54 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:53:55.148 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:54 smithi017 ceph-mon[126339]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:53:55.148 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:54 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:53:55.148 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:54 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:53:55.148 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:54 smithi017 ceph-mon[126339]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-14T11:53:55.148 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:54 smithi017 ceph-mon[126339]: monmap e2: 2 mons at {a=[v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0],c=[v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0]} removed_ranks: {} 2023-07-14T11:53:55.149 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:54 smithi017 ceph-mon[126339]: fsmap 2023-07-14T11:53:55.149 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:54 smithi017 ceph-mon[126339]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T11:53:55.149 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:54 smithi017 ceph-mon[126339]: mgrmap e13: x(active, since 49s) 2023-07-14T11:53:55.149 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:54 smithi017 ceph-mon[126339]: overall HEALTH_OK 2023-07-14T11:53:55.149 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:54 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:55.149 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:54 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:55.149 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:54 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:55.149 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:54 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T11:53:55.150 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:54 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:53:55.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.210+0000 7f7f60d41700 1 -- 172.21.15.97:0/3734533916 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f440035c0 msgr2=0x7f7f44005a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:55.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.210+0000 7f7f60d41700 1 --2- 172.21.15.97:0/3734533916 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f440035c0 0x7f7f44005a40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c0778e0 tx=0x7f7f5000feb0 comp rx=0 tx=0).stop 2023-07-14T11:53:55.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.210+0000 7f7f60d41700 1 -- 172.21.15.97:0/3734533916 shutdown_connections 2023-07-14T11:53:55.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.210+0000 7f7f60d41700 1 --2- 172.21.15.97:0/3734533916 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f440035c0 0x7f7f44005a40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:55.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.210+0000 7f7f60d41700 1 -- 172.21.15.97:0/3734533916 >> 172.21.15.97:0/3734533916 conn(0x7f7f5c073d70 msgr2=0x7f7f5c074170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:55.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.210+0000 7f7f60d41700 1 -- 172.21.15.97:0/3734533916 shutdown_connections 2023-07-14T11:53:55.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.210+0000 7f7f60d41700 1 -- 172.21.15.97:0/3734533916 wait complete. 2023-07-14T11:53:55.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.212+0000 7f7f60d41700 1 Processor -- start 2023-07-14T11:53:55.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.212+0000 7f7f60d41700 1 -- start start 2023-07-14T11:53:55.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.212+0000 7f7f60d41700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f440035c0 0x7f7f5c199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:55.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.212+0000 7f7f60d41700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f5c199e60 0x7f7f5c1939a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:55.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.212+0000 7f7f60d41700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5c19a390 con 0x7f7f5c199e60 2023-07-14T11:53:55.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.212+0000 7f7f60d41700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5c19a500 con 0x7f7f440035c0 2023-07-14T11:53:55.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.212+0000 7f7f5a59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f440035c0 0x7f7f5c199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:55.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.212+0000 7f7f5a59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f440035c0 0x7f7f5c199920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42140/0 (socket says 172.21.15.97:42140) 2023-07-14T11:53:55.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.212+0000 7f7f5a59c700 1 -- 172.21.15.97:0/3919238974 learned_addr learned my addr 172.21.15.97:0/3919238974 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-14T11:53:55.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.212+0000 7f7f5a59c700 1 -- 172.21.15.97:0/3919238974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f440035c0 msgr2=0x7f7f5c199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-07-14T11:53:55.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.212+0000 7f7f5a59c700 1 -- 172.21.15.97:0/3919238974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f440035c0 msgr2=0x7f7f5c199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-07-14T11:53:55.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.212+0000 7f7f5a59c700 1 --2- 172.21.15.97:0/3919238974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f440035c0 0x7f7f5c199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-14T11:53:55.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.212+0000 7f7f59d9b700 1 --2- 172.21.15.97:0/3919238974 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f5c199e60 0x7f7f5c1939a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:55.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.212+0000 7f7f5a59c700 1 --2- 172.21.15.97:0/3919238974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f440035c0 0x7f7f5c199920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-14T11:53:55.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.213+0000 7f7f59d9b700 1 -- 172.21.15.97:0/3919238974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f440035c0 msgr2=0x7f7f5c199920 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T11:53:55.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.213+0000 7f7f59d9b700 1 --2- 172.21.15.97:0/3919238974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f440035c0 0x7f7f5c199920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:55.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.213+0000 7f7f59d9b700 1 -- 172.21.15.97:0/3919238974 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f5000f900 con 0x7f7f5c199e60 2023-07-14T11:53:55.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.213+0000 7f7f59d9b700 1 --2- 172.21.15.97:0/3919238974 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f5c199e60 0x7f7f5c1939a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c00b500 tx=0x7f7f4c00b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:55.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.213+0000 7f7f4b7fe700 1 -- 172.21.15.97:0/3919238974 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f7f4c011630 con 0x7f7f5c199e60 2023-07-14T11:53:55.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.213+0000 7f7f4b7fe700 1 -- 172.21.15.97:0/3919238974 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f4c011c50 con 0x7f7f5c199e60 2023-07-14T11:53:55.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.213+0000 7f7f60d41700 1 -- 172.21.15.97:0/3919238974 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f5c193fa0 con 0x7f7f5c199e60 2023-07-14T11:53:55.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.213+0000 7f7f4b7fe700 1 -- 172.21.15.97:0/3919238974 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f7f4c010cc0 con 0x7f7f5c199e60 2023-07-14T11:53:55.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.213+0000 7f7f60d41700 1 -- 172.21.15.97:0/3919238974 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f5c1944f0 con 0x7f7f5c199e60 2023-07-14T11:53:55.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.214+0000 7f7f4b7fe700 1 -- 172.21.15.97:0/3919238974 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f7f4c01a410 con 0x7f7f5c199e60 2023-07-14T11:53:55.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.214+0000 7f7f4b7fe700 1 --2- 172.21.15.97:0/3919238974 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f7f3c0385f0 0x7f7f3c03aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:53:55.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.214+0000 7f7f4b7fe700 1 -- 172.21.15.97:0/3919238974 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7f4c04ac00 con 0x7f7f5c199e60 2023-07-14T11:53:55.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.215+0000 7f7f60d41700 1 -- 172.21.15.97:0/3919238974 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f5c1947d0 con 0x7f7f5c199e60 2023-07-14T11:53:55.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.215+0000 7f7f5a59c700 1 --2- 172.21.15.97:0/3919238974 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f7f3c0385f0 0x7f7f3c03aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:53:55.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.216+0000 7f7f5a59c700 1 --2- 172.21.15.97:0/3919238974 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f7f3c0385f0 0x7f7f3c03aaa0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7f500059d0 tx=0x7f7f50005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:53:55.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.220+0000 7f7f4b7fe700 1 -- 172.21.15.97:0/3919238974 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7f4c0193b0 con 0x7f7f5c199e60 2023-07-14T11:53:55.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.484+0000 7f7f60d41700 1 -- 172.21.15.97:0/3919238974 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7f5c194eb0 con 0x7f7f5c199e60 2023-07-14T11:53:55.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.485+0000 7f7f4b7fe700 1 -- 172.21.15.97:0/3919238974 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1002 (secure 0 0 0) 0x7f7f4c03e030 con 0x7f7f5c199e60 2023-07-14T11:53:55.467 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-14T11:53:55.467 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":2,"fsid":"b3c386cc-223c-11ee-9b33-001a4aab830c","modified":"2023-07-14T11:53:49.683566Z","created":"2023-07-14T11:51:56.547088Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:3300","nonce":0},{"type":"v1","addr":"172.21.15.97:6789","nonce":0}]},"addr":"172.21.15.97:6789/0","public_addr":"172.21.15.97:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-07-14T11:53:55.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.486+0000 7f7f60d41700 1 -- 172.21.15.97:0/3919238974 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f7f3c0385f0 msgr2=0x7f7f3c03aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:55.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.486+0000 7f7f60d41700 1 --2- 172.21.15.97:0/3919238974 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f7f3c0385f0 0x7f7f3c03aaa0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7f500059d0 tx=0x7f7f50005960 comp rx=0 tx=0).stop 2023-07-14T11:53:55.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.487+0000 7f7f60d41700 1 -- 172.21.15.97:0/3919238974 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f5c199e60 msgr2=0x7f7f5c1939a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:53:55.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.487+0000 7f7f60d41700 1 --2- 172.21.15.97:0/3919238974 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f5c199e60 0x7f7f5c1939a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c00b500 tx=0x7f7f4c00b8c0 comp rx=0 tx=0).stop 2023-07-14T11:53:55.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.487+0000 7f7f60d41700 1 -- 172.21.15.97:0/3919238974 shutdown_connections 2023-07-14T11:53:55.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.487+0000 7f7f60d41700 1 --2- 172.21.15.97:0/3919238974 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f7f3c0385f0 0x7f7f3c03aaa0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:55.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.487+0000 7f7f60d41700 1 --2- 172.21.15.97:0/3919238974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f440035c0 0x7f7f5c199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:55.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.487+0000 7f7f60d41700 1 --2- 172.21.15.97:0/3919238974 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f5c199e60 0x7f7f5c1939a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:53:55.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.487+0000 7f7f60d41700 1 -- 172.21.15.97:0/3919238974 >> 172.21.15.97:0/3919238974 conn(0x7f7f5c073d70 msgr2=0x7f7f5c1008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:53:55.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.487+0000 7f7f60d41700 1 -- 172.21.15.97:0/3919238974 shutdown_connections 2023-07-14T11:53:55.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:53:55.487+0000 7f7f60d41700 1 -- 172.21.15.97:0/3919238974 wait complete. 2023-07-14T11:53:55.471 INFO:teuthology.orchestra.run.smithi097.stderr:dumped monmap epoch 2 2023-07-14T11:53:56.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:55 smithi097 ceph-mon[129313]: Deploying daemon mon.b on smithi093 2023-07-14T11:53:56.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:55 smithi097 ceph-mon[129313]: from='client.? 172.21.15.97:0/3919238974' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-14T11:53:56.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:55 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:53:56.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:55 smithi017 ceph-mon[126339]: Deploying daemon mon.b on smithi093 2023-07-14T11:53:56.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:55 smithi017 ceph-mon[126339]: from='client.? 172.21.15.97:0/3919238974' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-14T11:53:56.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:55 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:53:56.866 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-07-14T11:53:56.866 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph mon dump -f json 2023-07-14T11:53:57.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:53:56 smithi097 ceph-mon[129313]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:53:57.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:53:56 smithi017 ceph-mon[126339]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:53:57.425 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T11:53:57.572 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 systemd[1]: Starting Ceph mon.b for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T11:53:57.857 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 podman[128882]: 2023-07-14T11:53:57.857 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: set uid:gid to 167:167 (ceph:ceph) 2023-07-14T11:53:57.857 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev), process ceph-mon, pid 7 2023-07-14T11:53:57.857 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 bash[128882]: 1030cf3d22ca0ff40283d379f0b34fc0f0fd3d28a15204e84180712c1e8c75e7 2023-07-14T11:53:57.858 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: pidfile_write: ignore empty --pid-file 2023-07-14T11:53:57.858 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: load: jerasure load: lrc 2023-07-14T11:53:57.858 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: RocksDB version: 7.9.2 2023-07-14T11:53:57.858 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Git sha 0 2023-07-14T11:53:57.858 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Compile date 2023-07-13 18:29:45 2023-07-14T11:53:57.858 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: DB SUMMARY 2023-07-14T11:53:57.858 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: DB Session ID: PMWQP2FLIY79EULVRAYK 2023-07-14T11:53:57.858 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: CURRENT file: CURRENT 2023-07-14T11:53:57.859 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: IDENTITY file: IDENTITY 2023-07-14T11:53:57.859 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-07-14T11:53:57.859 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-07-14T11:53:57.859 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-07-14T11:53:57.859 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.error_if_exists: 0 2023-07-14T11:53:57.859 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.create_if_missing: 0 2023-07-14T11:53:57.859 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.paranoid_checks: 1 2023-07-14T11:53:57.860 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.flush_verify_memtable_count: 1 2023-07-14T11:53:57.860 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-07-14T11:53:57.860 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-07-14T11:53:57.860 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.env: 0x55dddc243720 2023-07-14T11:53:57.860 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.fs: PosixFileSystem 2023-07-14T11:53:57.860 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.info_log: 0x55ddde7b5280 2023-07-14T11:53:57.860 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.max_file_opening_threads: 16 2023-07-14T11:53:57.861 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.statistics: (nil) 2023-07-14T11:53:57.861 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.use_fsync: 0 2023-07-14T11:53:57.861 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.max_log_file_size: 0 2023-07-14T11:53:57.861 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-07-14T11:53:57.861 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.log_file_time_to_roll: 0 2023-07-14T11:53:57.861 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.keep_log_file_num: 1000 2023-07-14T11:53:57.861 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.recycle_log_file_num: 0 2023-07-14T11:53:57.861 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.allow_fallocate: 1 2023-07-14T11:53:57.862 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.allow_mmap_reads: 0 2023-07-14T11:53:57.862 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.allow_mmap_writes: 0 2023-07-14T11:53:57.862 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.use_direct_reads: 0 2023-07-14T11:53:57.862 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-07-14T11:53:57.862 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.create_missing_column_families: 0 2023-07-14T11:53:57.862 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.db_log_dir: 2023-07-14T11:53:57.862 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.wal_dir: 2023-07-14T11:53:57.862 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.table_cache_numshardbits: 6 2023-07-14T11:53:57.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.WAL_ttl_seconds: 0 2023-07-14T11:53:57.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.WAL_size_limit_MB: 0 2023-07-14T11:53:57.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-07-14T11:53:57.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-07-14T11:53:57.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.is_fd_close_on_exec: 1 2023-07-14T11:53:57.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.advise_random_on_open: 1 2023-07-14T11:53:57.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.db_write_buffer_size: 0 2023-07-14T11:53:57.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.write_buffer_manager: 0x55ddde6d63c0 2023-07-14T11:53:57.864 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-07-14T11:53:57.864 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-07-14T11:53:57.864 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.use_adaptive_mutex: 0 2023-07-14T11:53:57.864 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.rate_limiter: (nil) 2023-07-14T11:53:57.864 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-07-14T11:53:57.864 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.wal_recovery_mode: 2 2023-07-14T11:53:57.864 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.enable_thread_tracking: 0 2023-07-14T11:53:57.865 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.enable_pipelined_write: 0 2023-07-14T11:53:57.865 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.unordered_write: 0 2023-07-14T11:53:57.865 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-07-14T11:53:57.865 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-07-14T11:53:57.865 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-07-14T11:53:57.865 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-07-14T11:53:57.865 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.row_cache: None 2023-07-14T11:53:57.865 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.wal_filter: None 2023-07-14T11:53:57.866 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-07-14T11:53:57.866 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.allow_ingest_behind: 0 2023-07-14T11:53:57.866 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.two_write_queues: 0 2023-07-14T11:53:57.866 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.manual_wal_flush: 0 2023-07-14T11:53:57.866 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.wal_compression: 0 2023-07-14T11:53:57.866 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.atomic_flush: 0 2023-07-14T11:53:57.866 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-07-14T11:53:57.866 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.persist_stats_to_disk: 0 2023-07-14T11:53:57.867 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.write_dbid_to_manifest: 0 2023-07-14T11:53:57.867 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.log_readahead_size: 0 2023-07-14T11:53:57.867 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-07-14T11:53:57.867 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.best_efforts_recovery: 0 2023-07-14T11:53:57.867 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-07-14T11:53:57.867 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-07-14T11:53:57.867 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.allow_data_in_errors: 0 2023-07-14T11:53:57.867 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.db_host_id: __hostname__ 2023-07-14T11:53:57.868 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.enforce_single_del_contracts: true 2023-07-14T11:53:57.868 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.max_background_jobs: 2 2023-07-14T11:53:57.868 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.max_background_compactions: -1 2023-07-14T11:53:57.868 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.max_subcompactions: 1 2023-07-14T11:53:57.868 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-07-14T11:53:57.868 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-07-14T11:53:57.868 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.delayed_write_rate : 16777216 2023-07-14T11:53:57.869 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.max_total_wal_size: 0 2023-07-14T11:53:57.869 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-07-14T11:53:57.869 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.stats_dump_period_sec: 600 2023-07-14T11:53:57.869 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.stats_persist_period_sec: 600 2023-07-14T11:53:57.869 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-07-14T11:53:57.869 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.max_open_files: -1 2023-07-14T11:53:57.869 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.bytes_per_sync: 0 2023-07-14T11:53:57.869 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.wal_bytes_per_sync: 0 2023-07-14T11:53:57.870 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.strict_bytes_per_sync: 0 2023-07-14T11:53:57.870 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.compaction_readahead_size: 0 2023-07-14T11:53:57.870 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.max_background_flushes: -1 2023-07-14T11:53:57.870 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Compression algorithms supported: 2023-07-14T11:53:57.870 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-07-14T11:53:57.870 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: kZSTD supported: 0 2023-07-14T11:53:57.870 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: kXpressCompression supported: 0 2023-07-14T11:53:57.870 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: kLZ4HCCompression supported: 1 2023-07-14T11:53:57.871 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: kZlibCompression supported: 1 2023-07-14T11:53:57.871 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: kSnappyCompression supported: 1 2023-07-14T11:53:57.871 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: kLZ4Compression supported: 1 2023-07-14T11:53:57.871 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: kBZip2Compression supported: 0 2023-07-14T11:53:57.871 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Fast CRC32 supported: Supported on x86 2023-07-14T11:53:57.871 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: DMutex implementation: pthread_mutex_t 2023-07-14T11:53:57.871 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-07-14T11:53:57.871 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-07-14T11:53:57.872 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-07-14T11:53:57.872 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.merge_operator: 2023-07-14T11:53:57.872 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.compaction_filter: None 2023-07-14T11:53:57.872 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.compaction_filter_factory: None 2023-07-14T11:53:57.872 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.sst_partitioner_factory: None 2023-07-14T11:53:57.872 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.memtable_factory: SkipListFactory 2023-07-14T11:53:57.872 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.table_factory: BlockBasedTable 2023-07-14T11:53:57.872 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ddde7b5380) 2023-07-14T11:53:57.873 INFO:journalctl@ceph.mon.b.smithi093.stdout: cache_index_and_filter_blocks: 1 2023-07-14T11:53:57.873 INFO:journalctl@ceph.mon.b.smithi093.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-07-14T11:53:57.873 INFO:journalctl@ceph.mon.b.smithi093.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-07-14T11:53:57.873 INFO:journalctl@ceph.mon.b.smithi093.stdout: pin_top_level_index_and_filter: 1 2023-07-14T11:53:57.873 INFO:journalctl@ceph.mon.b.smithi093.stdout: index_type: 0 2023-07-14T11:53:57.873 INFO:journalctl@ceph.mon.b.smithi093.stdout: data_block_index_type: 0 2023-07-14T11:53:57.873 INFO:journalctl@ceph.mon.b.smithi093.stdout: index_shortening: 1 2023-07-14T11:53:57.873 INFO:journalctl@ceph.mon.b.smithi093.stdout: data_block_hash_table_util_ratio: 0.750000 2023-07-14T11:53:57.874 INFO:journalctl@ceph.mon.b.smithi093.stdout: checksum: 4 2023-07-14T11:53:57.874 INFO:journalctl@ceph.mon.b.smithi093.stdout: no_block_cache: 0 2023-07-14T11:53:57.874 INFO:journalctl@ceph.mon.b.smithi093.stdout: block_cache: 0x55ddde7751f0 2023-07-14T11:53:57.874 INFO:journalctl@ceph.mon.b.smithi093.stdout: block_cache_name: BinnedLRUCache 2023-07-14T11:53:57.874 INFO:journalctl@ceph.mon.b.smithi093.stdout: block_cache_options: 2023-07-14T11:53:57.874 INFO:journalctl@ceph.mon.b.smithi093.stdout: capacity : 536870912 2023-07-14T11:53:57.874 INFO:journalctl@ceph.mon.b.smithi093.stdout: num_shard_bits : 4 2023-07-14T11:53:57.874 INFO:journalctl@ceph.mon.b.smithi093.stdout: strict_capacity_limit : 0 2023-07-14T11:53:57.875 INFO:journalctl@ceph.mon.b.smithi093.stdout: high_pri_pool_ratio: 0.000 2023-07-14T11:53:57.875 INFO:journalctl@ceph.mon.b.smithi093.stdout: block_cache_compressed: (nil) 2023-07-14T11:53:57.875 INFO:journalctl@ceph.mon.b.smithi093.stdout: persistent_cache: (nil) 2023-07-14T11:53:57.875 INFO:journalctl@ceph.mon.b.smithi093.stdout: block_size: 4096 2023-07-14T11:53:57.875 INFO:journalctl@ceph.mon.b.smithi093.stdout: block_size_deviation: 10 2023-07-14T11:53:57.875 INFO:journalctl@ceph.mon.b.smithi093.stdout: block_restart_interval: 16 2023-07-14T11:53:57.875 INFO:journalctl@ceph.mon.b.smithi093.stdout: index_block_restart_interval: 1 2023-07-14T11:53:57.875 INFO:journalctl@ceph.mon.b.smithi093.stdout: metadata_block_size: 4096 2023-07-14T11:53:57.876 INFO:journalctl@ceph.mon.b.smithi093.stdout: partition_filters: 0 2023-07-14T11:53:57.876 INFO:journalctl@ceph.mon.b.smithi093.stdout: use_delta_encoding: 1 2023-07-14T11:53:57.876 INFO:journalctl@ceph.mon.b.smithi093.stdout: filter_policy: bloomfilter 2023-07-14T11:53:57.876 INFO:journalctl@ceph.mon.b.smithi093.stdout: whole_key_filtering: 1 2023-07-14T11:53:57.876 INFO:journalctl@ceph.mon.b.smithi093.stdout: verify_compression: 0 2023-07-14T11:53:57.876 INFO:journalctl@ceph.mon.b.smithi093.stdout: read_amp_bytes_per_bit: 0 2023-07-14T11:53:57.876 INFO:journalctl@ceph.mon.b.smithi093.stdout: format_version: 5 2023-07-14T11:53:57.877 INFO:journalctl@ceph.mon.b.smithi093.stdout: enable_index_compression: 1 2023-07-14T11:53:57.877 INFO:journalctl@ceph.mon.b.smithi093.stdout: block_align: 0 2023-07-14T11:53:57.877 INFO:journalctl@ceph.mon.b.smithi093.stdout: max_auto_readahead_size: 262144 2023-07-14T11:53:57.877 INFO:journalctl@ceph.mon.b.smithi093.stdout: prepopulate_block_cache: 0 2023-07-14T11:53:57.877 INFO:journalctl@ceph.mon.b.smithi093.stdout: initial_auto_readahead_size: 8192 2023-07-14T11:53:57.877 INFO:journalctl@ceph.mon.b.smithi093.stdout: num_file_reads_for_auto_readahead: 2 2023-07-14T11:53:57.877 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.write_buffer_size: 33554432 2023-07-14T11:53:57.877 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.max_write_buffer_number: 2 2023-07-14T11:53:57.878 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.compression: NoCompression 2023-07-14T11:53:57.878 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.bottommost_compression: Disabled 2023-07-14T11:53:57.878 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.prefix_extractor: nullptr 2023-07-14T11:53:57.878 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-07-14T11:53:57.878 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.num_levels: 7 2023-07-14T11:53:57.878 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-07-14T11:53:57.878 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-07-14T11:53:57.878 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-07-14T11:53:57.879 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-07-14T11:53:57.879 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-07-14T11:53:57.879 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-07-14T11:53:57.880 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-07-14T11:53:57.880 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-07-14T11:53:57.880 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-07-14T11:53:57.880 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-07-14T11:53:57.881 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-07-14T11:53:57.881 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-07-14T11:53:57.881 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.compression_opts.window_bits: -14 2023-07-14T11:53:57.881 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.compression_opts.level: 32767 2023-07-14T11:53:57.881 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.compression_opts.strategy: 0 2023-07-14T11:53:57.881 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-07-14T11:53:57.881 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-07-14T11:53:57.882 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-07-14T11:53:57.882 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-07-14T11:53:57.882 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.compression_opts.enabled: false 2023-07-14T11:53:57.882 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-07-14T11:53:57.882 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-07-14T11:53:57.882 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-07-14T11:53:57.882 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-07-14T11:53:57.882 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.target_file_size_base: 67108864 2023-07-14T11:53:57.883 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.target_file_size_multiplier: 1 2023-07-14T11:53:57.883 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-07-14T11:53:57.883 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-07-14T11:53:57.883 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-07-14T11:53:57.883 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-07-14T11:53:57.883 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-07-14T11:53:57.883 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-07-14T11:53:57.883 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-07-14T11:53:57.884 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-07-14T11:53:57.884 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-07-14T11:53:57.884 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-07-14T11:53:57.884 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-07-14T11:53:57.884 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-07-14T11:53:57.884 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-07-14T11:53:57.884 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.arena_block_size: 1048576 2023-07-14T11:53:57.884 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-07-14T11:53:57.885 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-07-14T11:53:57.885 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.disable_auto_compactions: 0 2023-07-14T11:53:57.885 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-07-14T11:53:57.885 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-07-14T11:53:57.885 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-07-14T11:53:57.885 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-07-14T11:53:57.885 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-07-14T11:53:57.886 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-07-14T11:53:57.886 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-07-14T11:53:57.886 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-07-14T11:53:57.886 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-07-14T11:53:57.886 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-07-14T11:53:57.886 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.table_properties_collectors: 2023-07-14T11:53:57.886 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.inplace_update_support: 0 2023-07-14T11:53:57.886 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.inplace_update_num_locks: 10000 2023-07-14T11:53:57.887 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-07-14T11:53:57.887 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-07-14T11:53:57.887 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.memtable_huge_page_size: 0 2023-07-14T11:53:57.887 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.bloom_locality: 0 2023-07-14T11:53:57.887 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.max_successive_merges: 0 2023-07-14T11:53:57.887 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.optimize_filters_for_hits: 0 2023-07-14T11:53:57.887 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.paranoid_file_checks: 0 2023-07-14T11:53:57.887 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.force_consistency_checks: 1 2023-07-14T11:53:57.888 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.report_bg_io_stats: 0 2023-07-14T11:53:57.888 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.ttl: 2592000 2023-07-14T11:53:57.888 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.periodic_compaction_seconds: 0 2023-07-14T11:53:57.888 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-07-14T11:53:57.888 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-07-14T11:53:57.888 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.enable_blob_files: false 2023-07-14T11:53:57.888 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.min_blob_size: 0 2023-07-14T11:53:57.888 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.blob_file_size: 268435456 2023-07-14T11:53:57.889 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.blob_compression_type: NoCompression 2023-07-14T11:53:57.889 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.enable_blob_garbage_collection: false 2023-07-14T11:53:57.889 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-07-14T11:53:57.889 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-07-14T11:53:57.889 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-07-14T11:53:57.889 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.blob_file_starting_level: 0 2023-07-14T11:53:57.889 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-07-14T11:53:57.889 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-07-14T11:53:57.890 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-07-14T11:53:57.890 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e060a189-4a02-441f-b7dd-aa69671ff8c6 2023-07-14T11:53:57.890 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689335637800979, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-07-14T11:53:57.890 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-07-14T11:53:57.890 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689335637801518, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1689335637, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e060a189-4a02-441f-b7dd-aa69671ff8c6", "db_session_id": "PMWQP2FLIY79EULVRAYK", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-07-14T11:53:57.890 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689335637801607, "job": 1, "event": "recovery_finished"} 2023-07-14T11:53:57.890 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-07-14T11:53:57.891 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-07-14T11:53:57.891 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ddde80e000 2023-07-14T11:53:57.891 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: DB pointer 0x55ddde7fa000 2023-07-14T11:53:57.891 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-07-14T11:53:57.891 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-07-14T11:53:57.891 INFO:journalctl@ceph.mon.b.smithi093.stdout: ** DB Stats ** 2023-07-14T11:53:57.891 INFO:journalctl@ceph.mon.b.smithi093.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-14T11:53:57.891 INFO:journalctl@ceph.mon.b.smithi093.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-07-14T11:53:57.892 INFO:journalctl@ceph.mon.b.smithi093.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-14T11:53:57.892 INFO:journalctl@ceph.mon.b.smithi093.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-14T11:53:57.892 INFO:journalctl@ceph.mon.b.smithi093.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-07-14T11:53:57.892 INFO:journalctl@ceph.mon.b.smithi093.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-14T11:53:57.892 INFO:journalctl@ceph.mon.b.smithi093.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-14T11:53:57.892 INFO:journalctl@ceph.mon.b.smithi093.stdout: 2023-07-14T11:53:57.892 INFO:journalctl@ceph.mon.b.smithi093.stdout: ** Compaction Stats [default] ** 2023-07-14T11:53:57.892 INFO:journalctl@ceph.mon.b.smithi093.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-14T11:53:57.893 INFO:journalctl@ceph.mon.b.smithi093.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-07-14T11:53:57.893 INFO:journalctl@ceph.mon.b.smithi093.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-14T11:53:57.893 INFO:journalctl@ceph.mon.b.smithi093.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-14T11:53:57.893 INFO:journalctl@ceph.mon.b.smithi093.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-14T11:53:57.893 INFO:journalctl@ceph.mon.b.smithi093.stdout: 2023-07-14T11:53:57.893 INFO:journalctl@ceph.mon.b.smithi093.stdout: ** Compaction Stats [default] ** 2023-07-14T11:53:57.893 INFO:journalctl@ceph.mon.b.smithi093.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-14T11:53:57.893 INFO:journalctl@ceph.mon.b.smithi093.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-07-14T11:53:57.894 INFO:journalctl@ceph.mon.b.smithi093.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-14T11:53:57.894 INFO:journalctl@ceph.mon.b.smithi093.stdout: 2023-07-14T11:53:57.894 INFO:journalctl@ceph.mon.b.smithi093.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-07-14T11:53:57.894 INFO:journalctl@ceph.mon.b.smithi093.stdout: 2023-07-14T11:53:57.894 INFO:journalctl@ceph.mon.b.smithi093.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-14T11:53:57.894 INFO:journalctl@ceph.mon.b.smithi093.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-07-14T11:53:57.894 INFO:journalctl@ceph.mon.b.smithi093.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-07-14T11:53:57.895 INFO:journalctl@ceph.mon.b.smithi093.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-07-14T11:53:57.895 INFO:journalctl@ceph.mon.b.smithi093.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-07-14T11:53:57.895 INFO:journalctl@ceph.mon.b.smithi093.stdout: AddFile(Keys): cumulative 0, interval 0 2023-07-14T11:53:57.895 INFO:journalctl@ceph.mon.b.smithi093.stdout: Cumulative compaction: 0.00 GB write, 0.51 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-14T11:53:57.895 INFO:journalctl@ceph.mon.b.smithi093.stdout: Interval compaction: 0.00 GB write, 0.51 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-14T11:53:57.895 INFO:journalctl@ceph.mon.b.smithi093.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-07-14T11:53:57.895 INFO:journalctl@ceph.mon.b.smithi093.stdout: Block cache BinnedLRUCache@0x55ddde7751f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-07-14T11:53:57.895 INFO:journalctl@ceph.mon.b.smithi093.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-07-14T11:53:57.896 INFO:journalctl@ceph.mon.b.smithi093.stdout: 2023-07-14T11:53:57.896 INFO:journalctl@ceph.mon.b.smithi093.stdout: ** File Read Latency Histogram By Level [default] ** 2023-07-14T11:53:57.896 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-07-14T11:53:57.896 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: using public_addr v2:172.21.15.93:0/0 -> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] 2023-07-14T11:53:57.896 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: starting mon.b rank -1 at public addrs [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] at bind addrs [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid b3c386cc-223c-11ee-9b33-001a4aab830c 2023-07-14T11:53:57.896 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: mon.b@-1(???) e0 preinit fsid b3c386cc-223c-11ee-9b33-001a4aab830c 2023-07-14T11:53:57.896 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: mon.b@-1(synchronizing).mds e1 new map 2023-07-14T11:53:57.897 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: mon.b@-1(synchronizing).mds e1 print_map 2023-07-14T11:53:57.897 INFO:journalctl@ceph.mon.b.smithi093.stdout: e1 2023-07-14T11:53:57.897 INFO:journalctl@ceph.mon.b.smithi093.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-07-14T11:53:57.897 INFO:journalctl@ceph.mon.b.smithi093.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-14T11:53:57.897 INFO:journalctl@ceph.mon.b.smithi093.stdout: legacy client fscid: -1 2023-07-14T11:53:57.897 INFO:journalctl@ceph.mon.b.smithi093.stdout: 2023-07-14T11:53:57.897 INFO:journalctl@ceph.mon.b.smithi093.stdout: No filesystems configured 2023-07-14T11:53:57.897 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-07-14T11:53:57.898 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-07-14T11:53:57.898 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-07-14T11:53:57.898 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-07-14T11:53:57.898 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-07-14T11:53:57.898 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-07-14T11:53:57.898 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-07-14T11:53:57.898 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-14T11:53:57.898 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-14T11:53:57.899 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-14T11:53:57.899 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: Updating smithi093:/etc/ceph/ceph.conf 2023-07-14T11:53:57.899 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: Updating smithi093:/var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T11:53:57.899 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: Updating smithi093:/etc/ceph/ceph.client.admin.keyring 2023-07-14T11:53:57.899 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:57.899 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:57.899 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:57.899 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: Updating smithi093:/var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T11:53:57.900 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:53:57.900 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi097", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:53:57.900 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: Deploying cephadm binary to smithi097 2023-07-14T11:53:57.900 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:53:57.900 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:57.900 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:53:57.900 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:57.901 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: Added host smithi097 2023-07-14T11:53:57.901 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:57.901 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:53:57.901 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:57.901 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T11:53:57.901 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:53:57.901 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:57.901 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:57.902 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:57.902 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:57.902 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi097", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:53:57.902 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:53:57.902 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:53:57.902 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: Updating smithi097:/etc/ceph/ceph.conf 2023-07-14T11:53:57.902 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: Updating smithi097:/var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T11:53:57.902 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2023-07-14T11:53:57.903 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:53:57.903 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: Updating smithi097:/var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T11:53:57.903 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:57.903 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:57.903 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:57.903 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/109858388' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-07-14T11:53:57.903 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/109858388' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-07-14T11:53:57.903 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T11:53:57.904 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:53:57.904 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi017:172.21.15.17=a;smithi093:172.21.15.93=b;smithi097:172.21.15.97=c", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:53:57.904 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: Saving service mon spec with placement smithi017:172.21.15.17=a;smithi093:172.21.15.93=b;smithi097:172.21.15.97=c;count:3 2023-07-14T11:53:57.904 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:57.904 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:53:57.904 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:53:57.904 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:53:57.905 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:57.905 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T11:53:57.905 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:53:57.905 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: Deploying daemon mon.c on smithi097 2023-07-14T11:53:57.905 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:53:57.905 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:53:57.905 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:53:57.905 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T11:53:57.906 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:53:57.906 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: mon.a calling monitor election 2023-07-14T11:53:57.906 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:53:57.906 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:53:57.906 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:53:57.906 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: mon.c calling monitor election 2023-07-14T11:53:57.906 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:53:57.907 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:53:57.907 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:53:57.907 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:53:57.907 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-14T11:53:57.907 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: monmap e2: 2 mons at {a=[v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0],c=[v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0]} removed_ranks: {} 2023-07-14T11:53:57.907 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: fsmap 2023-07-14T11:53:57.907 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T11:53:57.907 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: mgrmap e13: x(active, since 49s) 2023-07-14T11:53:57.908 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: overall HEALTH_OK 2023-07-14T11:53:57.908 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:57.908 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:57.908 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:53:57.908 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T11:53:57.908 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:53:57.908 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: Deploying daemon mon.b on smithi093 2023-07-14T11:53:57.909 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: from='client.? 172.21.15.97:0/3919238974' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-14T11:53:57.909 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:53:57.909 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:53:57.909 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-07-14T11:53:57.909 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2023-07-14T11:53:57.909 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: expand_channel_meta expand map: {default=false} 2023-07-14T11:53:57.909 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: expand_channel_meta from 'false' to 'false' 2023-07-14T11:53:57.909 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: expand_channel_meta expanded map: {default=false} 2023-07-14T11:53:57.910 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: expand_channel_meta expand map: {default=info} 2023-07-14T11:53:57.910 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: expand_channel_meta from 'info' to 'info' 2023-07-14T11:53:57.910 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: expand_channel_meta expanded map: {default=info} 2023-07-14T11:53:57.910 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: expand_channel_meta expand map: {default=daemon} 2023-07-14T11:53:57.910 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: expand_channel_meta from 'daemon' to 'daemon' 2023-07-14T11:53:57.910 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: expand_channel_meta expanded map: {default=daemon} 2023-07-14T11:53:57.910 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: expand_channel_meta expand map: {default=debug} 2023-07-14T11:53:57.910 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: expand_channel_meta from 'debug' to 'debug' 2023-07-14T11:53:57.911 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 ceph-mon[128922]: expand_channel_meta expanded map: {default=debug} 2023-07-14T11:53:57.911 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:53:57 smithi093 systemd[1]: Started Ceph mon.b for b3c386cc-223c-11ee-9b33-001a4aab830c. 2023-07-14T11:54:03.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.103+0000 7f6334532700 1 -- 172.21.15.97:0/3742162342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f632c102df0 msgr2=0x7f6318007820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:03.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.103+0000 7f6334532700 1 --2- 172.21.15.97:0/3742162342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f632c102df0 0x7f6318007820 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f631c005fa0 tx=0x7f631c010f80 comp rx=0 tx=0).stop 2023-07-14T11:54:03.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.104+0000 7f6334532700 1 -- 172.21.15.97:0/3742162342 shutdown_connections 2023-07-14T11:54:03.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.104+0000 7f6334532700 1 --2- 172.21.15.97:0/3742162342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f632c102df0 0x7f6318007820 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:03.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.104+0000 7f6334532700 1 -- 172.21.15.97:0/3742162342 >> 172.21.15.97:0/3742162342 conn(0x7f632c073d70 msgr2=0x7f632c074170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:03.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.104+0000 7f6334532700 1 -- 172.21.15.97:0/3742162342 shutdown_connections 2023-07-14T11:54:03.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.104+0000 7f6334532700 1 -- 172.21.15.97:0/3742162342 wait complete. 2023-07-14T11:54:03.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.105+0000 7f6334532700 1 Processor -- start 2023-07-14T11:54:03.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.105+0000 7f6334532700 1 -- start start 2023-07-14T11:54:03.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.105+0000 7f6334532700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f632c102df0 0x7f632c196f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:03.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.105+0000 7f6334532700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f632c197490 0x7f632c19b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:03.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.105+0000 7f6334532700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f632c19be40 0x7f632c19c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:03.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.105+0000 7f6334532700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f632c197b60 con 0x7f632c197490 2023-07-14T11:54:03.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.105+0000 7f6334532700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f632c1a3bb0 con 0x7f632c19be40 2023-07-14T11:54:03.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.105+0000 7f6334532700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f632c1a3cf0 con 0x7f632c102df0 2023-07-14T11:54:03.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.106+0000 7f63322ce700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f632c102df0 0x7f632c196f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:03.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.106+0000 7f63322ce700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f632c102df0 0x7f632c196f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42168/0 (socket says 172.21.15.97:42168) 2023-07-14T11:54:03.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.106+0000 7f63322ce700 1 -- 172.21.15.97:0/1763882046 learned_addr learned my addr 172.21.15.97:0/1763882046 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-14T11:54:03.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.106+0000 7f63322ce700 1 -- 172.21.15.97:0/1763882046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f632c102df0 msgr2=0x7f632c196f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-07-14T11:54:03.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.106+0000 7f63322ce700 1 -- 172.21.15.97:0/1763882046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f632c102df0 msgr2=0x7f632c196f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-07-14T11:54:03.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.106+0000 7f6331acd700 1 --2- 172.21.15.97:0/1763882046 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f632c197490 0x7f632c19b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:03.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.106+0000 7f63322ce700 1 --2- 172.21.15.97:0/1763882046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f632c102df0 0x7f632c196f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-14T11:54:03.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.106+0000 7f63322ce700 1 --2- 172.21.15.97:0/1763882046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f632c102df0 0x7f632c196f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-14T11:54:03.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.106+0000 7f6332acf700 1 --2- 172.21.15.97:0/1763882046 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f632c19be40 0x7f632c19c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:03.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.106+0000 7f6332acf700 1 -- 172.21.15.97:0/1763882046 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f632c19be40 msgr2=0x7f632c19c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2023-07-14T11:54:03.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.106+0000 7f6332acf700 1 -- 172.21.15.97:0/1763882046 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f632c19be40 msgr2=0x7f632c19c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-07-14T11:54:03.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.106+0000 7f6332acf700 1 --2- 172.21.15.97:0/1763882046 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f632c19be40 0x7f632c19c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-14T11:54:03.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.106+0000 7f6332acf700 1 --2- 172.21.15.97:0/1763882046 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f632c19be40 0x7f632c19c2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-14T11:54:03.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.106+0000 7f6331acd700 1 -- 172.21.15.97:0/1763882046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f632c102df0 msgr2=0x7f632c196f50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T11:54:03.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.106+0000 7f6331acd700 1 --2- 172.21.15.97:0/1763882046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f632c102df0 0x7f632c196f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:03.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.106+0000 7f6331acd700 1 -- 172.21.15.97:0/1763882046 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f632c19be40 msgr2=0x7f632c19c2e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T11:54:03.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.106+0000 7f6331acd700 1 --2- 172.21.15.97:0/1763882046 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f632c19be40 0x7f632c19c2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:03.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.106+0000 7f6331acd700 1 -- 172.21.15.97:0/1763882046 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f631c010980 con 0x7f632c197490 2023-07-14T11:54:03.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.106+0000 7f6331acd700 1 --2- 172.21.15.97:0/1763882046 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f632c197490 0x7f632c19b900 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f632800b580 tx=0x7f632800b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:03.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.107+0000 7f63237fe700 1 -- 172.21.15.97:0/1763882046 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6328004230 con 0x7f632c197490 2023-07-14T11:54:03.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.107+0000 7f63237fe700 1 -- 172.21.15.97:0/1763882046 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6328004390 con 0x7f632c197490 2023-07-14T11:54:03.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.107+0000 7f63237fe700 1 -- 172.21.15.97:0/1763882046 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6328011a70 con 0x7f632c197490 2023-07-14T11:54:03.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.107+0000 7f6334532700 1 -- 172.21.15.97:0/1763882046 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f632c1a3fa0 con 0x7f632c197490 2023-07-14T11:54:03.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.107+0000 7f6334532700 1 -- 172.21.15.97:0/1763882046 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f632c1a43f0 con 0x7f632c197490 2023-07-14T11:54:03.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.108+0000 7f6334532700 1 -- 172.21.15.97:0/1763882046 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f632c061b40 con 0x7f632c197490 2023-07-14T11:54:03.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.108+0000 7f63237fe700 1 -- 172.21.15.97:0/1763882046 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f63280036a0 con 0x7f632c197490 2023-07-14T11:54:03.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.108+0000 7f63237fe700 1 --2- 172.21.15.97:0/1763882046 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f6310038c80 0x7f631003b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:03.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.108+0000 7f63237fe700 1 -- 172.21.15.97:0/1763882046 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f632804ad00 con 0x7f632c197490 2023-07-14T11:54:03.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.109+0000 7f63322ce700 1 --2- 172.21.15.97:0/1763882046 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f6310038c80 0x7f631003b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:03.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.109+0000 7f63322ce700 1 --2- 172.21.15.97:0/1763882046 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f6310038c80 0x7f631003b130 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f631c005850 tx=0x7f631c0103d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:03.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.114+0000 7f63237fe700 1 -- 172.21.15.97:0/1763882046 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6328004500 con 0x7f632c197490 2023-07-14T11:54:03.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:02 smithi097 ceph-mon[129313]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:03.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:02 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T11:54:03.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:02 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:54:03.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:02 smithi097 ceph-mon[129313]: mon.c calling monitor election 2023-07-14T11:54:03.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:02 smithi097 ceph-mon[129313]: mon.a calling monitor election 2023-07-14T11:54:03.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:02 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:54:03.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:02 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:54:03.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:02 smithi097 ceph-mon[129313]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:03.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:02 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:54:03.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:02 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:54:03.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:02 smithi097 ceph-mon[129313]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:03.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:02 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:54:03.139 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:02 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:54:03.139 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:02 smithi097 ceph-mon[129313]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-14T11:54:03.139 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:02 smithi097 ceph-mon[129313]: monmap e3: 3 mons at {a=[v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0],b=[v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0],c=[v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0]} removed_ranks: {} 2023-07-14T11:54:03.139 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:02 smithi097 ceph-mon[129313]: fsmap 2023-07-14T11:54:03.139 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:02 smithi097 ceph-mon[129313]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T11:54:03.139 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:02 smithi097 ceph-mon[129313]: mgrmap e13: x(active, since 57s) 2023-07-14T11:54:03.139 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:02 smithi097 ceph-mon[129313]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-07-14T11:54:03.140 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:02 smithi097 ceph-mon[129313]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-07-14T11:54:03.140 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:02 smithi097 ceph-mon[129313]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-07-14T11:54:03.140 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:02 smithi097 ceph-mon[129313]: mon.b (rank 2) addr [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] is down (out of quorum) 2023-07-14T11:54:03.140 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:02 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:03.140 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:02 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:03.140 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:02 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:03.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:02 smithi017 ceph-mon[126339]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:03.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:02 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T11:54:03.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:02 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:54:03.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:02 smithi017 ceph-mon[126339]: mon.c calling monitor election 2023-07-14T11:54:03.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:02 smithi017 ceph-mon[126339]: mon.a calling monitor election 2023-07-14T11:54:03.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:02 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:54:03.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:02 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:54:03.148 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:02 smithi017 ceph-mon[126339]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:03.148 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:02 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:54:03.148 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:02 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:54:03.148 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:02 smithi017 ceph-mon[126339]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:03.148 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:02 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:54:03.148 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:02 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:54:03.148 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:02 smithi017 ceph-mon[126339]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-14T11:54:03.148 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:02 smithi017 ceph-mon[126339]: monmap e3: 3 mons at {a=[v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0],b=[v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0],c=[v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0]} removed_ranks: {} 2023-07-14T11:54:03.149 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:02 smithi017 ceph-mon[126339]: fsmap 2023-07-14T11:54:03.149 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:02 smithi017 ceph-mon[126339]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T11:54:03.149 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:02 smithi017 ceph-mon[126339]: mgrmap e13: x(active, since 57s) 2023-07-14T11:54:03.149 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:02 smithi017 ceph-mon[126339]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-07-14T11:54:03.149 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:02 smithi017 ceph-mon[126339]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-07-14T11:54:03.149 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:02 smithi017 ceph-mon[126339]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-07-14T11:54:03.149 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:02 smithi017 ceph-mon[126339]: mon.b (rank 2) addr [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] is down (out of quorum) 2023-07-14T11:54:03.149 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:02 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:03.150 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:02 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:03.150 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:02 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:03.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.397+0000 7f6334532700 1 -- 172.21.15.97:0/1763882046 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f632c1a4770 con 0x7f632c197490 2023-07-14T11:54:03.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.398+0000 7f63237fe700 1 -- 172.21.15.97:0/1763882046 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1266 (secure 0 0 0) 0x7f6328021710 con 0x7f632c197490 2023-07-14T11:54:03.382 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-07-14T11:54:03.382 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":3,"fsid":"b3c386cc-223c-11ee-9b33-001a4aab830c","modified":"2023-07-14T11:53:57.834799Z","created":"2023-07-14T11:51:56.547088Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:3300","nonce":0},{"type":"v1","addr":"172.21.15.97:6789","nonce":0}]},"addr":"172.21.15.97:6789/0","public_addr":"172.21.15.97:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:3300","nonce":0},{"type":"v1","addr":"172.21.15.93:6789","nonce":0}]},"addr":"172.21.15.93:6789/0","public_addr":"172.21.15.93:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-07-14T11:54:03.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.400+0000 7f6334532700 1 -- 172.21.15.97:0/1763882046 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f6310038c80 msgr2=0x7f631003b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:03.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.400+0000 7f6334532700 1 --2- 172.21.15.97:0/1763882046 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f6310038c80 0x7f631003b130 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f631c005850 tx=0x7f631c0103d0 comp rx=0 tx=0).stop 2023-07-14T11:54:03.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.400+0000 7f6334532700 1 -- 172.21.15.97:0/1763882046 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f632c197490 msgr2=0x7f632c19b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:03.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.400+0000 7f6334532700 1 --2- 172.21.15.97:0/1763882046 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f632c197490 0x7f632c19b900 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f632800b580 tx=0x7f632800b940 comp rx=0 tx=0).stop 2023-07-14T11:54:03.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.400+0000 7f6334532700 1 -- 172.21.15.97:0/1763882046 shutdown_connections 2023-07-14T11:54:03.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.400+0000 7f6334532700 1 --2- 172.21.15.97:0/1763882046 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f6310038c80 0x7f631003b130 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:03.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.400+0000 7f6334532700 1 --2- 172.21.15.97:0/1763882046 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f632c19be40 0x7f632c19c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:03.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.400+0000 7f6334532700 1 --2- 172.21.15.97:0/1763882046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f632c102df0 0x7f632c196f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:03.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.400+0000 7f6334532700 1 --2- 172.21.15.97:0/1763882046 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f632c197490 0x7f632c19b900 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:03.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.400+0000 7f6334532700 1 -- 172.21.15.97:0/1763882046 >> 172.21.15.97:0/1763882046 conn(0x7f632c073d70 msgr2=0x7f632c18a890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:03.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.400+0000 7f6334532700 1 -- 172.21.15.97:0/1763882046 shutdown_connections 2023-07-14T11:54:03.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:03.400+0000 7f6334532700 1 -- 172.21.15.97:0/1763882046 wait complete. 2023-07-14T11:54:03.385 INFO:teuthology.orchestra.run.smithi097.stderr:dumped monmap epoch 3 2023-07-14T11:54:03.903 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-07-14T11:54:03.904 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph config generate-minimal-conf 2023-07-14T11:54:04.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:03 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:04.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:03 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:54:04.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:03 smithi097 ceph-mon[129313]: from='client.? 172.21.15.97:0/1763882046' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-14T11:54:04.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:03 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:54:04.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:03 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:04.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:03 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:54:04.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:03 smithi017 ceph-mon[126339]: from='client.? 172.21.15.97:0/1763882046' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-14T11:54:04.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:03 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:54:04.644 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T11:54:05.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:04 smithi093 ceph-mon[128922]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:05.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:04 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T11:54:05.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:04 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:54:05.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:04 smithi093 ceph-mon[128922]: mon.c calling monitor election 2023-07-14T11:54:05.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:04 smithi093 ceph-mon[128922]: mon.a calling monitor election 2023-07-14T11:54:05.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:04 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:54:05.115 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:04 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:54:05.115 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:04 smithi093 ceph-mon[128922]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:05.115 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:04 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:54:05.115 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:04 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:54:05.115 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:04 smithi093 ceph-mon[128922]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:05.115 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:04 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:54:05.115 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:04 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:54:05.116 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:04 smithi093 ceph-mon[128922]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-14T11:54:05.116 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:04 smithi093 ceph-mon[128922]: monmap e3: 3 mons at {a=[v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0],b=[v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0],c=[v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0]} removed_ranks: {} 2023-07-14T11:54:05.116 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:04 smithi093 ceph-mon[128922]: fsmap 2023-07-14T11:54:05.116 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:04 smithi093 ceph-mon[128922]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T11:54:05.116 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:04 smithi093 ceph-mon[128922]: mgrmap e13: x(active, since 57s) 2023-07-14T11:54:05.116 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:04 smithi093 ceph-mon[128922]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-07-14T11:54:05.116 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:04 smithi093 ceph-mon[128922]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-07-14T11:54:05.116 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:04 smithi093 ceph-mon[128922]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-07-14T11:54:05.117 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:04 smithi093 ceph-mon[128922]: mon.b (rank 2) addr [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] is down (out of quorum) 2023-07-14T11:54:05.117 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:04 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:05.117 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:04 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:05.117 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:04 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:05.117 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:04 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:05.117 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:04 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:54:05.117 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:04 smithi093 ceph-mon[128922]: from='client.? 172.21.15.97:0/1763882046' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-14T11:54:05.118 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:04 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:54:06.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:05 smithi093 ceph-mon[128922]: mon.b calling monitor election 2023-07-14T11:54:06.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:05 smithi093 ceph-mon[128922]: Updating smithi017:/etc/ceph/ceph.conf 2023-07-14T11:54:06.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:05 smithi093 ceph-mon[128922]: Updating smithi093:/etc/ceph/ceph.conf 2023-07-14T11:54:06.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:05 smithi093 ceph-mon[128922]: Updating smithi097:/etc/ceph/ceph.conf 2023-07-14T11:54:06.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:05 smithi093 ceph-mon[128922]: mon.c calling monitor election 2023-07-14T11:54:06.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:05 smithi093 ceph-mon[128922]: mon.b calling monitor election 2023-07-14T11:54:06.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:05 smithi093 ceph-mon[128922]: mon.a calling monitor election 2023-07-14T11:54:06.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:05 smithi093 ceph-mon[128922]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-07-14T11:54:06.115 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:05 smithi093 ceph-mon[128922]: monmap e3: 3 mons at {a=[v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0],b=[v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0],c=[v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0]} removed_ranks: {} 2023-07-14T11:54:06.115 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:05 smithi093 ceph-mon[128922]: fsmap 2023-07-14T11:54:06.115 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:05 smithi093 ceph-mon[128922]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T11:54:06.115 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:05 smithi093 ceph-mon[128922]: mgrmap e13: x(active, since 59s) 2023-07-14T11:54:06.115 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:05 smithi093 ceph-mon[128922]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-07-14T11:54:06.116 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:05 smithi093 ceph-mon[128922]: Cluster is now healthy 2023-07-14T11:54:06.116 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:05 smithi093 ceph-mon[128922]: overall HEALTH_OK 2023-07-14T11:54:06.116 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:05 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:06.116 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:05 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:06.116 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:05 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:06.116 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:05 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:06.116 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:05 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:06.124 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:05 smithi017 ceph-mon[126339]: mon.b calling monitor election 2023-07-14T11:54:06.124 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:05 smithi017 ceph-mon[126339]: Updating smithi017:/etc/ceph/ceph.conf 2023-07-14T11:54:06.125 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:05 smithi017 ceph-mon[126339]: Updating smithi093:/etc/ceph/ceph.conf 2023-07-14T11:54:06.125 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:05 smithi017 ceph-mon[126339]: Updating smithi097:/etc/ceph/ceph.conf 2023-07-14T11:54:06.125 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:05 smithi017 ceph-mon[126339]: mon.c calling monitor election 2023-07-14T11:54:06.126 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:05 smithi017 ceph-mon[126339]: mon.b calling monitor election 2023-07-14T11:54:06.126 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:05 smithi017 ceph-mon[126339]: mon.a calling monitor election 2023-07-14T11:54:06.126 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:05 smithi017 ceph-mon[126339]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-07-14T11:54:06.126 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:05 smithi017 ceph-mon[126339]: monmap e3: 3 mons at {a=[v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0],b=[v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0],c=[v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0]} removed_ranks: {} 2023-07-14T11:54:06.127 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:05 smithi017 ceph-mon[126339]: fsmap 2023-07-14T11:54:06.127 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:05 smithi017 ceph-mon[126339]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T11:54:06.127 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:05 smithi017 ceph-mon[126339]: mgrmap e13: x(active, since 59s) 2023-07-14T11:54:06.127 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:05 smithi017 ceph-mon[126339]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-07-14T11:54:06.128 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:05 smithi017 ceph-mon[126339]: Cluster is now healthy 2023-07-14T11:54:06.128 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:05 smithi017 ceph-mon[126339]: overall HEALTH_OK 2023-07-14T11:54:06.128 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:05 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:06.129 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:05 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:06.129 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:05 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:06.129 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:05 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:06.130 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:05 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:06.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:05 smithi097 ceph-mon[129313]: mon.b calling monitor election 2023-07-14T11:54:06.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:05 smithi097 ceph-mon[129313]: Updating smithi017:/etc/ceph/ceph.conf 2023-07-14T11:54:06.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:05 smithi097 ceph-mon[129313]: Updating smithi093:/etc/ceph/ceph.conf 2023-07-14T11:54:06.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:05 smithi097 ceph-mon[129313]: Updating smithi097:/etc/ceph/ceph.conf 2023-07-14T11:54:06.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:05 smithi097 ceph-mon[129313]: mon.c calling monitor election 2023-07-14T11:54:06.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:05 smithi097 ceph-mon[129313]: mon.b calling monitor election 2023-07-14T11:54:06.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:05 smithi097 ceph-mon[129313]: mon.a calling monitor election 2023-07-14T11:54:06.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:05 smithi097 ceph-mon[129313]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-07-14T11:54:06.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:05 smithi097 ceph-mon[129313]: monmap e3: 3 mons at {a=[v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0],b=[v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0],c=[v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0]} removed_ranks: {} 2023-07-14T11:54:06.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:05 smithi097 ceph-mon[129313]: fsmap 2023-07-14T11:54:06.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:05 smithi097 ceph-mon[129313]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T11:54:06.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:05 smithi097 ceph-mon[129313]: mgrmap e13: x(active, since 59s) 2023-07-14T11:54:06.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:05 smithi097 ceph-mon[129313]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-07-14T11:54:06.139 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:05 smithi097 ceph-mon[129313]: Cluster is now healthy 2023-07-14T11:54:06.139 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:05 smithi097 ceph-mon[129313]: overall HEALTH_OK 2023-07-14T11:54:06.139 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:05 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:06.139 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:05 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:06.139 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:05 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:06.139 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:05 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:06.139 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:05 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:06.383 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.400+0000 7f4346077700 1 -- 172.21.15.17:0/1978484830 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4340105630 msgr2=0x7f4340105a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:06.384 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.400+0000 7f4346077700 1 --2- 172.21.15.17:0/1978484830 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4340105630 0x7f4340105a00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4330009910 tx=0x7f4330009c20 comp rx=0 tx=0).stop 2023-07-14T11:54:06.384 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.400+0000 7f4346077700 1 -- 172.21.15.17:0/1978484830 shutdown_connections 2023-07-14T11:54:06.384 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.400+0000 7f4346077700 1 --2- 172.21.15.17:0/1978484830 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4340105630 0x7f4340105a00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:06.384 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.400+0000 7f4346077700 1 -- 172.21.15.17:0/1978484830 >> 172.21.15.17:0/1978484830 conn(0x7f43400fb130 msgr2=0x7f43400fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:06.384 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.400+0000 7f4346077700 1 -- 172.21.15.17:0/1978484830 shutdown_connections 2023-07-14T11:54:06.385 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.400+0000 7f4346077700 1 -- 172.21.15.17:0/1978484830 wait complete. 2023-07-14T11:54:06.385 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.401+0000 7f4346077700 1 Processor -- start 2023-07-14T11:54:06.385 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.402+0000 7f4346077700 1 -- start start 2023-07-14T11:54:06.385 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.402+0000 7f4346077700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4340105630 0x7f434019a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:06.386 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.402+0000 7f4346077700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f434019ae30 0x7f4340194970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:06.386 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.402+0000 7f4346077700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4340194eb0 0x7f4340195350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:06.386 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.402+0000 7f4346077700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f434019b440 con 0x7f4340194eb0 2023-07-14T11:54:06.386 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.402+0000 7f4346077700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43401a1710 con 0x7f4340105630 2023-07-14T11:54:06.386 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.402+0000 7f4346077700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43401a1850 con 0x7f434019ae30 2023-07-14T11:54:06.387 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.402+0000 7f433ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4340194eb0 0x7f4340195350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:06.387 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.402+0000 7f433ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4340194eb0 0x7f4340195350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50996/0 (socket says 172.21.15.17:50996) 2023-07-14T11:54:06.387 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.402+0000 7f433ffff700 1 -- 172.21.15.17:0/2401911076 learned_addr learned my addr 172.21.15.17:0/2401911076 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:54:06.388 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.402+0000 7f433f7fe700 1 --2- 172.21.15.17:0/2401911076 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4340105630 0x7f434019a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:06.388 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.403+0000 7f433effd700 1 --2- 172.21.15.17:0/2401911076 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f434019ae30 0x7f4340194970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:06.388 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.403+0000 7f433ffff700 1 -- 172.21.15.17:0/2401911076 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f434019ae30 msgr2=0x7f4340194970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:06.388 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.403+0000 7f433ffff700 1 --2- 172.21.15.17:0/2401911076 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f434019ae30 0x7f4340194970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:06.388 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.403+0000 7f433ffff700 1 -- 172.21.15.17:0/2401911076 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4340105630 msgr2=0x7f434019a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:06.388 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.403+0000 7f433ffff700 1 --2- 172.21.15.17:0/2401911076 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4340105630 0x7f434019a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:06.388 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.403+0000 7f433ffff700 1 -- 172.21.15.17:0/2401911076 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43300095f0 con 0x7f4340194eb0 2023-07-14T11:54:06.389 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.403+0000 7f433f7fe700 1 --2- 172.21.15.17:0/2401911076 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4340105630 0x7f434019a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:54:06.389 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.403+0000 7f433ffff700 1 --2- 172.21.15.17:0/2401911076 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4340194eb0 0x7f4340195350 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f433800ba50 tx=0x7f433800bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:06.389 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.403+0000 7f433cff9700 1 -- 172.21.15.17:0/2401911076 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433800c930 con 0x7f4340194eb0 2023-07-14T11:54:06.389 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.403+0000 7f433cff9700 1 -- 172.21.15.17:0/2401911076 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4338014440 con 0x7f4340194eb0 2023-07-14T11:54:06.389 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.403+0000 7f4346077700 1 -- 172.21.15.17:0/2401911076 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43401a1b30 con 0x7f4340194eb0 2023-07-14T11:54:06.389 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.404+0000 7f433cff9700 1 -- 172.21.15.17:0/2401911076 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43380075b0 con 0x7f4340194eb0 2023-07-14T11:54:06.389 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.404+0000 7f4346077700 1 -- 172.21.15.17:0/2401911076 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43401a2030 con 0x7f4340194eb0 2023-07-14T11:54:06.390 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.405+0000 7f433cff9700 1 -- 172.21.15.17:0/2401911076 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f4338007710 con 0x7f4340194eb0 2023-07-14T11:54:06.390 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.406+0000 7f433cff9700 1 --2- 172.21.15.17:0/2401911076 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f4328038c30 0x7f432803b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:06.390 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.406+0000 7f4346077700 1 -- 172.21.15.17:0/2401911076 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4340061b40 con 0x7f4340194eb0 2023-07-14T11:54:06.391 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.406+0000 7f433cff9700 1 -- 172.21.15.17:0/2401911076 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4338050660 con 0x7f4340194eb0 2023-07-14T11:54:06.391 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.406+0000 7f433f7fe700 1 --2- 172.21.15.17:0/2401911076 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f4328038c30 0x7f432803b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:06.391 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.407+0000 7f433f7fe700 1 --2- 172.21.15.17:0/2401911076 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f4328038c30 0x7f432803b0e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4330006010 tx=0x7f433000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:06.394 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.411+0000 7f433cff9700 1 -- 172.21.15.17:0/2401911076 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4338019070 con 0x7f4340194eb0 2023-07-14T11:54:06.607 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.624+0000 7f4346077700 1 -- 172.21.15.17:0/2401911076 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f4340195e80 con 0x7f4340194eb0 2023-07-14T11:54:06.608 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.624+0000 7f433cff9700 1 -- 172.21.15.17:0/2401911076 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+271 (secure 0 0 0) 0x7f4338051060 con 0x7f4340194eb0 2023-07-14T11:54:06.608 INFO:teuthology.orchestra.run.smithi017.stdout:# minimal ceph.conf for b3c386cc-223c-11ee-9b33-001a4aab830c 2023-07-14T11:54:06.608 INFO:teuthology.orchestra.run.smithi017.stdout:[global] 2023-07-14T11:54:06.608 INFO:teuthology.orchestra.run.smithi017.stdout: fsid = b3c386cc-223c-11ee-9b33-001a4aab830c 2023-07-14T11:54:06.608 INFO:teuthology.orchestra.run.smithi017.stdout: mon_host = [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] 2023-07-14T11:54:06.610 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.627+0000 7f4346077700 1 -- 172.21.15.17:0/2401911076 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f4328038c30 msgr2=0x7f432803b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:06.611 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.627+0000 7f4346077700 1 --2- 172.21.15.17:0/2401911076 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f4328038c30 0x7f432803b0e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4330006010 tx=0x7f433000b540 comp rx=0 tx=0).stop 2023-07-14T11:54:06.611 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.627+0000 7f4346077700 1 -- 172.21.15.17:0/2401911076 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4340194eb0 msgr2=0x7f4340195350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:06.611 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.627+0000 7f4346077700 1 --2- 172.21.15.17:0/2401911076 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4340194eb0 0x7f4340195350 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f433800ba50 tx=0x7f433800bd60 comp rx=0 tx=0).stop 2023-07-14T11:54:06.611 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.627+0000 7f4346077700 1 -- 172.21.15.17:0/2401911076 shutdown_connections 2023-07-14T11:54:06.611 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.627+0000 7f4346077700 1 --2- 172.21.15.17:0/2401911076 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f4328038c30 0x7f432803b0e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:06.611 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.627+0000 7f4346077700 1 --2- 172.21.15.17:0/2401911076 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4340105630 0x7f434019a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:06.611 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.627+0000 7f4346077700 1 --2- 172.21.15.17:0/2401911076 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4340194eb0 0x7f4340195350 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:06.612 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.627+0000 7f4346077700 1 --2- 172.21.15.17:0/2401911076 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f434019ae30 0x7f4340194970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:06.612 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.627+0000 7f4346077700 1 -- 172.21.15.17:0/2401911076 >> 172.21.15.17:0/2401911076 conn(0x7f43400fb130 msgr2=0x7f43400fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:06.612 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.627+0000 7f4346077700 1 -- 172.21.15.17:0/2401911076 shutdown_connections 2023-07-14T11:54:06.612 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:06.627+0000 7f4346077700 1 -- 172.21.15.17:0/2401911076 wait complete. 2023-07-14T11:54:06.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:06 smithi017 ceph-mon[126339]: Updating smithi093:/var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T11:54:06.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:06 smithi017 ceph-mon[126339]: Updating smithi097:/var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T11:54:06.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:06 smithi017 ceph-mon[126339]: Updating smithi017:/var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T11:54:06.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:06 smithi017 ceph-mon[126339]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:06.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:06 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:54:06.898 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:06 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:06.898 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:06 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:06.898 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:06 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:06.898 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:06 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T11:54:06.898 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:06 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-14T11:54:06.898 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:06 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:06.898 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:06 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/2401911076' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:07.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:06 smithi093 ceph-mon[128922]: Updating smithi093:/var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T11:54:07.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:06 smithi093 ceph-mon[128922]: Updating smithi097:/var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T11:54:07.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:06 smithi093 ceph-mon[128922]: Updating smithi017:/var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T11:54:07.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:06 smithi093 ceph-mon[128922]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:07.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:06 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:54:07.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:06 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:07.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:06 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:07.115 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:06 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:07.115 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:06 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T11:54:07.115 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:06 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-14T11:54:07.115 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:06 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:07.115 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:06 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/2401911076' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:07.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:06 smithi097 ceph-mon[129313]: Updating smithi093:/var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T11:54:07.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:06 smithi097 ceph-mon[129313]: Updating smithi097:/var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T11:54:07.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:06 smithi097 ceph-mon[129313]: Updating smithi017:/var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T11:54:07.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:06 smithi097 ceph-mon[129313]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:07.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:06 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:54:07.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:06 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:07.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:06 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:07.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:06 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:07.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:06 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T11:54:07.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:06 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-14T11:54:07.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:06 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:07.139 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:06 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/2401911076' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:07.230 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-07-14T11:54:07.230 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2023-07-14T11:54:07.231 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/etc/ceph/ceph.conf 2023-07-14T11:54:07.309 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2023-07-14T11:54:07.309 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-14T11:54:07.420 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-07-14T11:54:07.420 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/etc/ceph/ceph.conf 2023-07-14T11:54:07.458 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-07-14T11:54:07.459 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-14T11:54:07.531 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-14T11:54:07.532 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.conf 2023-07-14T11:54:07.568 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-14T11:54:07.568 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-14T11:54:07.640 INFO:tasks.cephadm:Adding mgr.x on smithi017 2023-07-14T11:54:07.641 INFO:tasks.cephadm:Adding mgr.y on smithi093 2023-07-14T11:54:07.641 INFO:tasks.cephadm:Adding mgr.z on smithi097 2023-07-14T11:54:07.642 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph orch apply mgr '3;smithi017=x;smithi093=y;smithi097=z' 2023-07-14T11:54:07.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:07 smithi097 ceph-mon[129313]: Reconfiguring mon.a (unknown last config time)... 2023-07-14T11:54:07.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:07 smithi097 ceph-mon[129313]: Reconfiguring daemon mon.a on smithi017 2023-07-14T11:54:07.896 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:07 smithi017 ceph-mon[126339]: Reconfiguring mon.a (unknown last config time)... 2023-07-14T11:54:07.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:07 smithi017 ceph-mon[126339]: Reconfiguring daemon mon.a on smithi017 2023-07-14T11:54:08.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:07 smithi093 ceph-mon[128922]: Reconfiguring mon.a (unknown last config time)... 2023-07-14T11:54:08.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:07 smithi093 ceph-mon[128922]: Reconfiguring daemon mon.a on smithi017 2023-07-14T11:54:08.239 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T11:54:09.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:08 smithi093 ceph-mon[128922]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:09.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:08 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:09.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:08 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:09.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:08 smithi093 ceph-mon[128922]: Reconfiguring mon.b (monmap changed)... 2023-07-14T11:54:09.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:08 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T11:54:09.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:08 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-14T11:54:09.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:08 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:09.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:08 smithi093 ceph-mon[128922]: Reconfiguring daemon mon.b on smithi093 2023-07-14T11:54:09.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:08 smithi097 ceph-mon[129313]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:09.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:08 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:09.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:08 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:09.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:08 smithi097 ceph-mon[129313]: Reconfiguring mon.b (monmap changed)... 2023-07-14T11:54:09.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:08 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T11:54:09.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:08 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-14T11:54:09.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:08 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:09.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:08 smithi097 ceph-mon[129313]: Reconfiguring daemon mon.b on smithi093 2023-07-14T11:54:09.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:08 smithi017 ceph-mon[126339]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:09.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:08 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:09.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:08 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:09.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:08 smithi017 ceph-mon[126339]: Reconfiguring mon.b (monmap changed)... 2023-07-14T11:54:09.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:08 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T11:54:09.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:08 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-14T11:54:09.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:08 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:09.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:08 smithi017 ceph-mon[126339]: Reconfiguring daemon mon.b on smithi093 2023-07-14T11:54:09.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.503+0000 7f79bbeeb700 1 -- 172.21.15.97:0/4045727630 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79b40ffc90 msgr2=0x7f79b41000a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:09.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.503+0000 7f79bbeeb700 1 --2- 172.21.15.97:0/4045727630 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79b40ffc90 0x7f79b41000a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f79a4009910 tx=0x7f79a4009c20 comp rx=0 tx=0).stop 2023-07-14T11:54:09.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.504+0000 7f79bbeeb700 1 -- 172.21.15.97:0/4045727630 shutdown_connections 2023-07-14T11:54:09.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.504+0000 7f79bbeeb700 1 --2- 172.21.15.97:0/4045727630 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79b40ffc90 0x7f79b41000a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:09.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.504+0000 7f79bbeeb700 1 -- 172.21.15.97:0/4045727630 >> 172.21.15.97:0/4045727630 conn(0x7f79b40f9320 msgr2=0x7f79b40fb770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:09.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.504+0000 7f79bbeeb700 1 -- 172.21.15.97:0/4045727630 shutdown_connections 2023-07-14T11:54:09.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.504+0000 7f79bbeeb700 1 -- 172.21.15.97:0/4045727630 wait complete. 2023-07-14T11:54:09.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.504+0000 7f79bbeeb700 1 Processor -- start 2023-07-14T11:54:09.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.504+0000 7f79bbeeb700 1 -- start start 2023-07-14T11:54:09.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.504+0000 7f79bbeeb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79b40ffc90 0x7f79b40fe310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:09.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.504+0000 7f79bbeeb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f79b4197730 0x7f79b4199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:09.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.504+0000 7f79bbeeb700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79b419a0f0 0x7f79b419c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:09.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.504+0000 7f79bbeeb700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b419cd10 con 0x7f79b419a0f0 2023-07-14T11:54:09.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.504+0000 7f79bbeeb700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b419ce80 con 0x7f79b4197730 2023-07-14T11:54:09.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.504+0000 7f79bbeeb700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b419d040 con 0x7f79b40ffc90 2023-07-14T11:54:09.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.504+0000 7f79b9c87700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79b40ffc90 0x7f79b40fe310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:09.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.505+0000 7f79b9c87700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79b40ffc90 0x7f79b40fe310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42178/0 (socket says 172.21.15.97:42178) 2023-07-14T11:54:09.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.505+0000 7f79b9c87700 1 -- 172.21.15.97:0/1063100654 learned_addr learned my addr 172.21.15.97:0/1063100654 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-14T11:54:09.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.505+0000 7f79ba488700 1 --2- 172.21.15.97:0/1063100654 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79b419a0f0 0x7f79b419c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:09.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.505+0000 7f79b9486700 1 --2- 172.21.15.97:0/1063100654 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f79b4197730 0x7f79b4199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:09.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.505+0000 7f79b9c87700 1 -- 172.21.15.97:0/1063100654 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f79b4197730 msgr2=0x7f79b4199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:09.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.505+0000 7f79b9c87700 1 --2- 172.21.15.97:0/1063100654 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f79b4197730 0x7f79b4199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:09.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.505+0000 7f79b9c87700 1 -- 172.21.15.97:0/1063100654 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79b419a0f0 msgr2=0x7f79b419c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:09.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.505+0000 7f79b9c87700 1 --2- 172.21.15.97:0/1063100654 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79b419a0f0 0x7f79b419c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:09.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.505+0000 7f79b9c87700 1 -- 172.21.15.97:0/1063100654 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a40095f0 con 0x7f79b40ffc90 2023-07-14T11:54:09.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.505+0000 7f79b9c87700 1 --2- 172.21.15.97:0/1063100654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79b40ffc90 0x7f79b40fe310 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f79a4005b40 tx=0x7f79a4004cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:09.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.505+0000 7f79aaffd700 1 -- 172.21.15.97:0/1063100654 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a401d070 con 0x7f79b40ffc90 2023-07-14T11:54:09.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.505+0000 7f79bbeeb700 1 -- 172.21.15.97:0/1063100654 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79b419d2c0 con 0x7f79b40ffc90 2023-07-14T11:54:09.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.505+0000 7f79aaffd700 1 -- 172.21.15.97:0/1063100654 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79a400be20 con 0x7f79b40ffc90 2023-07-14T11:54:09.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.505+0000 7f79aaffd700 1 -- 172.21.15.97:0/1063100654 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a4022d90 con 0x7f79b40ffc90 2023-07-14T11:54:09.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.505+0000 7f79bbeeb700 1 -- 172.21.15.97:0/1063100654 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79b419d7e0 con 0x7f79b40ffc90 2023-07-14T11:54:09.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.506+0000 7f79aaffd700 1 -- 172.21.15.97:0/1063100654 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f79a400f480 con 0x7f79b40ffc90 2023-07-14T11:54:09.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.506+0000 7f79bbeeb700 1 -- 172.21.15.97:0/1063100654 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79b418d290 con 0x7f79b40ffc90 2023-07-14T11:54:09.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.506+0000 7f79aaffd700 1 --2- 172.21.15.97:0/1063100654 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f79a0038c80 0x7f79a003b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:09.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.506+0000 7f79aaffd700 1 -- 172.21.15.97:0/1063100654 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f79a404ca90 con 0x7f79b40ffc90 2023-07-14T11:54:09.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.507+0000 7f79b9486700 1 --2- 172.21.15.97:0/1063100654 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f79a0038c80 0x7f79a003b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:09.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.507+0000 7f79b9486700 1 --2- 172.21.15.97:0/1063100654 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f79a0038c80 0x7f79a003b130 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f79b0009850 tx=0x7f79b0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:09.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.511+0000 7f79aaffd700 1 -- 172.21.15.97:0/1063100654 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f79a4020370 con 0x7f79b40ffc90 2023-07-14T11:54:09.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.716+0000 7f79bbeeb700 1 -- 172.21.15.97:0/1063100654 --> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi017=x;smithi093=y;smithi097=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f79b4060950 con 0x7f79a0038c80 2023-07-14T11:54:09.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.723+0000 7f79aaffd700 1 -- 172.21.15.97:0/1063100654 <== mgr.14152 v2:172.21.15.17:6800/4237449858 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f79b4060950 con 0x7f79a0038c80 2023-07-14T11:54:09.707 INFO:teuthology.orchestra.run.smithi097.stdout:Scheduled mgr update... 2023-07-14T11:54:09.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.726+0000 7f79bbeeb700 1 -- 172.21.15.97:0/1063100654 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f79a0038c80 msgr2=0x7f79a003b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:09.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.726+0000 7f79bbeeb700 1 --2- 172.21.15.97:0/1063100654 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f79a0038c80 0x7f79a003b130 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f79b0009850 tx=0x7f79b0008040 comp rx=0 tx=0).stop 2023-07-14T11:54:09.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.726+0000 7f79bbeeb700 1 -- 172.21.15.97:0/1063100654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79b40ffc90 msgr2=0x7f79b40fe310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:09.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.726+0000 7f79bbeeb700 1 --2- 172.21.15.97:0/1063100654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79b40ffc90 0x7f79b40fe310 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f79a4005b40 tx=0x7f79a4004cb0 comp rx=0 tx=0).stop 2023-07-14T11:54:09.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.726+0000 7f79bbeeb700 1 -- 172.21.15.97:0/1063100654 shutdown_connections 2023-07-14T11:54:09.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.726+0000 7f79bbeeb700 1 --2- 172.21.15.97:0/1063100654 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f79a0038c80 0x7f79a003b130 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:09.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.726+0000 7f79bbeeb700 1 --2- 172.21.15.97:0/1063100654 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79b419a0f0 0x7f79b419c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:09.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.726+0000 7f79bbeeb700 1 --2- 172.21.15.97:0/1063100654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79b40ffc90 0x7f79b40fe310 secure :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f79a4005b40 tx=0x7f79a4004cb0 comp rx=0 tx=0).stop 2023-07-14T11:54:09.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.726+0000 7f79bbeeb700 1 --2- 172.21.15.97:0/1063100654 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f79b4197730 0x7f79b4199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:09.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.726+0000 7f79bbeeb700 1 -- 172.21.15.97:0/1063100654 >> 172.21.15.97:0/1063100654 conn(0x7f79b40f9320 msgr2=0x7f79b40f9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:09.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.726+0000 7f79bbeeb700 1 -- 172.21.15.97:0/1063100654 shutdown_connections 2023-07-14T11:54:09.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:54:09.726+0000 7f79bbeeb700 1 -- 172.21.15.97:0/1063100654 wait complete. 2023-07-14T11:54:10.291 DEBUG:teuthology.orchestra.run.smithi093:mgr.y> sudo journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mgr.y.service 2023-07-14T11:54:10.295 DEBUG:teuthology.orchestra.run.smithi097:mgr.z> sudo journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mgr.z.service 2023-07-14T11:54:10.298 INFO:tasks.cephadm:Deploying OSDs... 2023-07-14T11:54:10.298 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2023-07-14T11:54:10.299 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/scratch_devs of=/dev/stdout 2023-07-14T11:54:10.327 INFO:journalctl@ceph.mgr.z.smithi097.stdout:-- Logs begin at Fri 2023-07-14 11:24:16 UTC. -- 2023-07-14T11:54:10.329 INFO:journalctl@ceph.mgr.y.smithi093.stdout:-- Logs begin at Fri 2023-07-14 11:24:16 UTC. -- 2023-07-14T11:54:10.348 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-14T11:54:10.348 DEBUG:teuthology.orchestra.run.smithi017:> stat /dev/vg_nvme/lv_1 2023-07-14T11:54:10.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:10 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:10.364 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:10 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:10.364 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:10 smithi093 ceph-mon[128922]: Reconfiguring mon.c (monmap changed)... 2023-07-14T11:54:10.364 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:10 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T11:54:10.365 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:10 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-14T11:54:10.365 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:10 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:10.365 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:10 smithi093 ceph-mon[128922]: Reconfiguring daemon mon.c on smithi097 2023-07-14T11:54:10.365 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:10 smithi093 ceph-mon[128922]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:10.366 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:10 smithi093 ceph-mon[128922]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi017=x;smithi093=y;smithi097=z", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:54:10.366 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:10 smithi093 ceph-mon[128922]: Saving service mgr spec with placement smithi017=x;smithi093=y;smithi097=z;count:3 2023-07-14T11:54:10.366 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:10 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:10.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:10 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:10.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:10 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:10.388 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:10 smithi097 ceph-mon[129313]: Reconfiguring mon.c (monmap changed)... 2023-07-14T11:54:10.388 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:10 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T11:54:10.388 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:10 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-14T11:54:10.389 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:10 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:10.389 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:10 smithi097 ceph-mon[129313]: Reconfiguring daemon mon.c on smithi097 2023-07-14T11:54:10.389 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:10 smithi097 ceph-mon[129313]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:10.389 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:10 smithi097 ceph-mon[129313]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi017=x;smithi093=y;smithi097=z", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:54:10.390 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:10 smithi097 ceph-mon[129313]: Saving service mgr spec with placement smithi017=x;smithi093=y;smithi097=z;count:3 2023-07-14T11:54:10.390 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:10 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:10.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:10 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:10.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:10 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:10.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:10 smithi017 ceph-mon[126339]: Reconfiguring mon.c (monmap changed)... 2023-07-14T11:54:10.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:10 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T11:54:10.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:10 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-14T11:54:10.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:10 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:10.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:10 smithi017 ceph-mon[126339]: Reconfiguring daemon mon.c on smithi097 2023-07-14T11:54:10.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:10 smithi017 ceph-mon[126339]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:10.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:10 smithi017 ceph-mon[126339]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi017=x;smithi093=y;smithi097=z", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:54:10.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:10 smithi017 ceph-mon[126339]: Saving service mgr spec with placement smithi017=x;smithi093=y;smithi097=z;count:3 2023-07-14T11:54:10.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:10 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:10.438 INFO:teuthology.orchestra.run.smithi017.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-14T11:54:10.438 INFO:teuthology.orchestra.run.smithi017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T11:54:10.438 INFO:teuthology.orchestra.run.smithi017.stdout:Device: 5h/5d Inode: 450 Links: 1 2023-07-14T11:54:10.438 INFO:teuthology.orchestra.run.smithi017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T11:54:10.438 INFO:teuthology.orchestra.run.smithi017.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T11:54:10.438 INFO:teuthology.orchestra.run.smithi017.stdout:Access: 2023-07-14 11:53:17.908486990 +0000 2023-07-14T11:54:10.439 INFO:teuthology.orchestra.run.smithi017.stdout:Modify: 2023-07-14 11:43:44.130255122 +0000 2023-07-14T11:54:10.439 INFO:teuthology.orchestra.run.smithi017.stdout:Change: 2023-07-14 11:43:44.130255122 +0000 2023-07-14T11:54:10.439 INFO:teuthology.orchestra.run.smithi017.stdout: Birth: 2023-07-14 11:43:44.105255434 +0000 2023-07-14T11:54:10.439 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-14T11:54:10.548 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records in 2023-07-14T11:54:10.549 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records out 2023-07-14T11:54:10.549 INFO:teuthology.orchestra.run.smithi017.stderr:512 bytes copied, 0.000185418 s, 2.8 MB/s 2023-07-14T11:54:10.551 DEBUG:teuthology.orchestra.run.smithi017:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-14T11:54:10.598 DEBUG:teuthology.orchestra.run.smithi017:> stat /dev/vg_nvme/lv_2 2023-07-14T11:54:10.644 INFO:teuthology.orchestra.run.smithi017.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-14T11:54:10.644 INFO:teuthology.orchestra.run.smithi017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T11:54:10.645 INFO:teuthology.orchestra.run.smithi017.stdout:Device: 5h/5d Inode: 456 Links: 1 2023-07-14T11:54:10.645 INFO:teuthology.orchestra.run.smithi017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T11:54:10.645 INFO:teuthology.orchestra.run.smithi017.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T11:54:10.645 INFO:teuthology.orchestra.run.smithi017.stdout:Access: 2023-07-14 11:53:17.908486990 +0000 2023-07-14T11:54:10.646 INFO:teuthology.orchestra.run.smithi017.stdout:Modify: 2023-07-14 11:43:44.969244646 +0000 2023-07-14T11:54:10.646 INFO:teuthology.orchestra.run.smithi017.stdout:Change: 2023-07-14 11:43:44.969244646 +0000 2023-07-14T11:54:10.646 INFO:teuthology.orchestra.run.smithi017.stdout: Birth: 2023-07-14 11:43:44.947244920 +0000 2023-07-14T11:54:10.647 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-14T11:54:10.753 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records in 2023-07-14T11:54:10.753 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records out 2023-07-14T11:54:10.754 INFO:teuthology.orchestra.run.smithi017.stderr:512 bytes copied, 0.000200282 s, 2.6 MB/s 2023-07-14T11:54:10.756 DEBUG:teuthology.orchestra.run.smithi017:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-14T11:54:10.802 DEBUG:teuthology.orchestra.run.smithi017:> stat /dev/vg_nvme/lv_3 2023-07-14T11:54:10.891 INFO:teuthology.orchestra.run.smithi017.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-14T11:54:10.891 INFO:teuthology.orchestra.run.smithi017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T11:54:10.892 INFO:teuthology.orchestra.run.smithi017.stdout:Device: 5h/5d Inode: 462 Links: 1 2023-07-14T11:54:10.892 INFO:teuthology.orchestra.run.smithi017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T11:54:10.892 INFO:teuthology.orchestra.run.smithi017.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T11:54:10.892 INFO:teuthology.orchestra.run.smithi017.stdout:Access: 2023-07-14 11:53:17.908486990 +0000 2023-07-14T11:54:10.892 INFO:teuthology.orchestra.run.smithi017.stdout:Modify: 2023-07-14 11:43:45.818234045 +0000 2023-07-14T11:54:10.892 INFO:teuthology.orchestra.run.smithi017.stdout:Change: 2023-07-14 11:43:45.818234045 +0000 2023-07-14T11:54:10.892 INFO:teuthology.orchestra.run.smithi017.stdout: Birth: 2023-07-14 11:43:45.796234320 +0000 2023-07-14T11:54:10.893 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-14T11:54:10.999 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records in 2023-07-14T11:54:11.000 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records out 2023-07-14T11:54:11.000 INFO:teuthology.orchestra.run.smithi017.stderr:512 bytes copied, 0.000211514 s, 2.4 MB/s 2023-07-14T11:54:11.002 DEBUG:teuthology.orchestra.run.smithi017:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-14T11:54:11.049 DEBUG:teuthology.orchestra.run.smithi017:> stat /dev/vg_nvme/lv_4 2023-07-14T11:54:11.093 INFO:teuthology.orchestra.run.smithi017.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-14T11:54:11.093 INFO:teuthology.orchestra.run.smithi017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T11:54:11.093 INFO:teuthology.orchestra.run.smithi017.stdout:Device: 5h/5d Inode: 468 Links: 1 2023-07-14T11:54:11.094 INFO:teuthology.orchestra.run.smithi017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T11:54:11.094 INFO:teuthology.orchestra.run.smithi017.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T11:54:11.094 INFO:teuthology.orchestra.run.smithi017.stdout:Access: 2023-07-14 11:53:17.908486990 +0000 2023-07-14T11:54:11.094 INFO:teuthology.orchestra.run.smithi017.stdout:Modify: 2023-07-14 11:43:46.659223545 +0000 2023-07-14T11:54:11.094 INFO:teuthology.orchestra.run.smithi017.stdout:Change: 2023-07-14 11:43:46.659223545 +0000 2023-07-14T11:54:11.094 INFO:teuthology.orchestra.run.smithi017.stdout: Birth: 2023-07-14 11:43:46.638223807 +0000 2023-07-14T11:54:11.095 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-14T11:54:11.201 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records in 2023-07-14T11:54:11.201 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records out 2023-07-14T11:54:11.201 INFO:teuthology.orchestra.run.smithi017.stderr:512 bytes copied, 0.000213343 s, 2.4 MB/s 2023-07-14T11:54:11.203 DEBUG:teuthology.orchestra.run.smithi017:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-14T11:54:11.249 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-07-14T11:54:11.249 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/scratch_devs of=/dev/stdout 2023-07-14T11:54:11.270 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-14T11:54:11.270 DEBUG:teuthology.orchestra.run.smithi093:> stat /dev/vg_nvme/lv_1 2023-07-14T11:54:11.330 INFO:teuthology.orchestra.run.smithi093.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-14T11:54:11.331 INFO:teuthology.orchestra.run.smithi093.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T11:54:11.331 INFO:teuthology.orchestra.run.smithi093.stdout:Device: 6h/6d Inode: 379076 Links: 1 2023-07-14T11:54:11.331 INFO:teuthology.orchestra.run.smithi093.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T11:54:11.331 INFO:teuthology.orchestra.run.smithi093.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T11:54:11.331 INFO:teuthology.orchestra.run.smithi093.stdout:Access: 2023-07-14 11:53:57.242649225 +0000 2023-07-14T11:54:11.331 INFO:teuthology.orchestra.run.smithi093.stdout:Modify: 2023-07-14 11:40:51.157429949 +0000 2023-07-14T11:54:11.332 INFO:teuthology.orchestra.run.smithi093.stdout:Change: 2023-07-14 11:40:51.157429949 +0000 2023-07-14T11:54:11.332 INFO:teuthology.orchestra.run.smithi093.stdout: Birth: - 2023-07-14T11:54:11.332 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-14T11:54:11.401 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records in 2023-07-14T11:54:11.402 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records out 2023-07-14T11:54:11.402 INFO:teuthology.orchestra.run.smithi093.stderr:512 bytes copied, 0.000270294 s, 1.9 MB/s 2023-07-14T11:54:11.403 DEBUG:teuthology.orchestra.run.smithi093:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-14T11:54:11.463 DEBUG:teuthology.orchestra.run.smithi093:> stat /dev/vg_nvme/lv_2 2023-07-14T11:54:11.521 INFO:teuthology.orchestra.run.smithi093.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-14T11:54:11.521 INFO:teuthology.orchestra.run.smithi093.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T11:54:11.522 INFO:teuthology.orchestra.run.smithi093.stdout:Device: 6h/6d Inode: 379388 Links: 1 2023-07-14T11:54:11.522 INFO:teuthology.orchestra.run.smithi093.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T11:54:11.522 INFO:teuthology.orchestra.run.smithi093.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T11:54:11.522 INFO:teuthology.orchestra.run.smithi093.stdout:Access: 2023-07-14 11:53:57.242649225 +0000 2023-07-14T11:54:11.522 INFO:teuthology.orchestra.run.smithi093.stdout:Modify: 2023-07-14 11:40:51.678417385 +0000 2023-07-14T11:54:11.522 INFO:teuthology.orchestra.run.smithi093.stdout:Change: 2023-07-14 11:40:51.678417385 +0000 2023-07-14T11:54:11.522 INFO:teuthology.orchestra.run.smithi093.stdout: Birth: - 2023-07-14T11:54:11.523 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-14T11:54:11.589 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records in 2023-07-14T11:54:11.590 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records out 2023-07-14T11:54:11.590 INFO:teuthology.orchestra.run.smithi093.stderr:512 bytes copied, 0.000229547 s, 2.2 MB/s 2023-07-14T11:54:11.591 DEBUG:teuthology.orchestra.run.smithi093:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-14T11:54:11.649 DEBUG:teuthology.orchestra.run.smithi093:> stat /dev/vg_nvme/lv_3 2023-07-14T11:54:11.707 INFO:teuthology.orchestra.run.smithi093.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-14T11:54:11.707 INFO:teuthology.orchestra.run.smithi093.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T11:54:11.707 INFO:teuthology.orchestra.run.smithi093.stdout:Device: 6h/6d Inode: 379606 Links: 1 2023-07-14T11:54:11.708 INFO:teuthology.orchestra.run.smithi093.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T11:54:11.708 INFO:teuthology.orchestra.run.smithi093.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T11:54:11.708 INFO:teuthology.orchestra.run.smithi093.stdout:Access: 2023-07-14 11:53:57.242649225 +0000 2023-07-14T11:54:11.708 INFO:teuthology.orchestra.run.smithi093.stdout:Modify: 2023-07-14 11:40:52.217404387 +0000 2023-07-14T11:54:11.708 INFO:teuthology.orchestra.run.smithi093.stdout:Change: 2023-07-14 11:40:52.217404387 +0000 2023-07-14T11:54:11.708 INFO:teuthology.orchestra.run.smithi093.stdout: Birth: - 2023-07-14T11:54:11.709 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-14T11:54:11.770 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:11 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:11.771 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:11 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:11.771 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:11 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:54:11.771 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:11 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:11.771 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:11 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:54:11.771 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:11 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:11.772 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:11 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-14T11:54:11.772 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:11 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-14T11:54:11.772 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:11 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-14T11:54:11.772 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:11 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:11.773 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records in 2023-07-14T11:54:11.773 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records out 2023-07-14T11:54:11.773 INFO:teuthology.orchestra.run.smithi093.stderr:512 bytes copied, 0.000227512 s, 2.3 MB/s 2023-07-14T11:54:11.774 DEBUG:teuthology.orchestra.run.smithi093:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-14T11:54:11.831 DEBUG:teuthology.orchestra.run.smithi093:> stat /dev/vg_nvme/lv_4 2023-07-14T11:54:11.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:11 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:11.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:11 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:11.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:11 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:54:11.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:11 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:11.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:11 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:54:11.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:11 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:11.888 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:11 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-14T11:54:11.888 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:11 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-14T11:54:11.888 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:11 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-14T11:54:11.888 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:11 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:11.889 INFO:teuthology.orchestra.run.smithi093.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-14T11:54:11.889 INFO:teuthology.orchestra.run.smithi093.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T11:54:11.889 INFO:teuthology.orchestra.run.smithi093.stdout:Device: 6h/6d Inode: 381246 Links: 1 2023-07-14T11:54:11.889 INFO:teuthology.orchestra.run.smithi093.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T11:54:11.890 INFO:teuthology.orchestra.run.smithi093.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T11:54:11.890 INFO:teuthology.orchestra.run.smithi093.stdout:Access: 2023-07-14 11:53:57.242649225 +0000 2023-07-14T11:54:11.890 INFO:teuthology.orchestra.run.smithi093.stdout:Modify: 2023-07-14 11:40:52.733391943 +0000 2023-07-14T11:54:11.890 INFO:teuthology.orchestra.run.smithi093.stdout:Change: 2023-07-14 11:40:52.733391943 +0000 2023-07-14T11:54:11.890 INFO:teuthology.orchestra.run.smithi093.stdout: Birth: - 2023-07-14T11:54:11.891 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-14T11:54:11.896 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:11 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:11.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:11 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:11.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:11 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:54:11.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:11 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:11.898 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:11 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:54:11.898 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:11 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:11.898 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:11 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-14T11:54:11.898 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:11 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-14T11:54:11.899 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:11 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-14T11:54:11.899 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:11 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:11.955 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records in 2023-07-14T11:54:11.955 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records out 2023-07-14T11:54:11.955 INFO:teuthology.orchestra.run.smithi093.stderr:512 bytes copied, 0.000192377 s, 2.7 MB/s 2023-07-14T11:54:11.956 DEBUG:teuthology.orchestra.run.smithi093:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-14T11:54:12.014 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-07-14T11:54:12.015 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/scratch_devs of=/dev/stdout 2023-07-14T11:54:12.031 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-14T11:54:12.032 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_1 2023-07-14T11:54:12.089 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-14T11:54:12.089 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T11:54:12.089 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 6h/6d Inode: 383002 Links: 1 2023-07-14T11:54:12.089 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T11:54:12.089 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T11:54:12.090 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2023-07-14 11:53:48.849120109 +0000 2023-07-14T11:54:12.090 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2023-07-14 11:47:00.854144902 +0000 2023-07-14T11:54:12.090 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2023-07-14 11:47:00.854144902 +0000 2023-07-14T11:54:12.090 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: - 2023-07-14T11:54:12.090 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-14T11:54:12.155 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2023-07-14T11:54:12.156 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2023-07-14T11:54:12.156 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.0002997 s, 1.7 MB/s 2023-07-14T11:54:12.157 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-14T11:54:12.214 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_2 2023-07-14T11:54:12.273 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-14T11:54:12.273 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T11:54:12.273 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 6h/6d Inode: 383153 Links: 1 2023-07-14T11:54:12.273 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T11:54:12.273 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T11:54:12.273 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2023-07-14 11:53:48.849120109 +0000 2023-07-14T11:54:12.274 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2023-07-14 11:47:01.394134540 +0000 2023-07-14T11:54:12.274 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2023-07-14 11:47:01.394134540 +0000 2023-07-14T11:54:12.274 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: - 2023-07-14T11:54:12.274 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-14T11:54:12.338 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2023-07-14T11:54:12.338 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2023-07-14T11:54:12.338 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.00019631 s, 2.6 MB/s 2023-07-14T11:54:12.339 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-14T11:54:12.397 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_3 2023-07-14T11:54:12.454 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-14T11:54:12.455 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T11:54:12.455 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 6h/6d Inode: 382229 Links: 1 2023-07-14T11:54:12.455 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T11:54:12.455 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T11:54:12.455 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2023-07-14 11:53:48.849120109 +0000 2023-07-14T11:54:12.455 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2023-07-14 11:47:01.904124753 +0000 2023-07-14T11:54:12.455 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2023-07-14 11:47:01.904124753 +0000 2023-07-14T11:54:12.455 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: - 2023-07-14T11:54:12.456 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-14T11:54:12.520 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2023-07-14T11:54:12.521 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2023-07-14T11:54:12.521 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000242137 s, 2.1 MB/s 2023-07-14T11:54:12.522 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-14T11:54:12.580 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_4 2023-07-14T11:54:12.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:12 smithi097 ceph-mon[129313]: Deploying daemon mgr.z on smithi097 2023-07-14T11:54:12.637 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:12 smithi097 ceph-mon[129313]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:12.638 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-14T11:54:12.638 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T11:54:12.638 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 6h/6d Inode: 376108 Links: 1 2023-07-14T11:54:12.638 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T11:54:12.638 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T11:54:12.638 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2023-07-14 11:53:48.849120109 +0000 2023-07-14T11:54:12.639 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2023-07-14 11:47:02.458114122 +0000 2023-07-14T11:54:12.639 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2023-07-14 11:47:02.458114122 +0000 2023-07-14T11:54:12.639 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: - 2023-07-14T11:54:12.639 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-14T11:54:12.744 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2023-07-14T11:54:12.745 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2023-07-14T11:54:12.745 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000194438 s, 2.6 MB/s 2023-07-14T11:54:12.746 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-14T11:54:12.763 INFO:tasks.cephadm:Deploying osd.0 on smithi017 with /dev/vg_nvme/lv_4... 2023-07-14T11:54:12.764 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-07-14T11:54:12.831 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:12 smithi017 ceph-mon[126339]: Deploying daemon mgr.z on smithi097 2023-07-14T11:54:12.831 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:12 smithi017 ceph-mon[126339]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:12.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:12 smithi093 ceph-mon[128922]: Deploying daemon mgr.z on smithi097 2023-07-14T11:54:12.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:12 smithi093 ceph-mon[128922]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:13.137 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Jul 14 11:54:12 smithi097 systemd[1]: Starting Ceph mgr.z for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T11:54:13.456 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Jul 14 11:54:13 smithi097 podman[131398]: 2023-07-14T11:54:13.457 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Jul 14 11:54:13 smithi097 bash[131398]: 67ac8386d724ca9a22c83c2395e99f4a8cfe2db78823c916aa45d943065c4407 2023-07-14T11:54:13.457 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Jul 14 11:54:13 smithi097 ceph-mgr[131437]: -- 172.21.15.97:0/3366428076 <== mon.1 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5653e626f1e0 con 0x5653e6245000 2023-07-14T11:54:13.500 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T11:54:13.720 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Jul 14 11:54:13 smithi097 systemd[1]: Started Ceph mgr.z for b3c386cc-223c-11ee-9b33-001a4aab830c. 2023-07-14T11:54:13.721 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Jul 14 11:54:13 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-z[131413]: 2023-07-14T11:54:13.571+0000 7fe76058e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-14T11:54:13.721 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Jul 14 11:54:13 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-z[131413]: 2023-07-14T11:54:13.649+0000 7fe76058e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-14T11:54:13.721 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Jul 14 11:54:13 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-z[131413]: 2023-07-14T11:54:13.737+0000 7fe76058e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-14T11:54:13.978 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Jul 14 11:54:13 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-z[131413]: 2023-07-14T11:54:13.829+0000 7fe76058e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-14T11:54:14.283 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Jul 14 11:54:13 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-z[131413]: 2023-07-14T11:54:13.995+0000 7fe76058e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-14T11:54:14.592 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Jul 14 11:54:14 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-z[131413]: 2023-07-14T11:54:14.300+0000 7fe76058e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-14T11:54:14.593 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Jul 14 11:54:14 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-z[131413]: 2023-07-14T11:54:14.387+0000 7fe76058e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-14T11:54:14.593 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Jul 14 11:54:14 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-z[131413]: 2023-07-14T11:54:14.515+0000 7fe76058e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-14T11:54:14.594 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:14 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:14.594 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:14 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:14.594 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:14 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:14.594 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:14 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-14T11:54:14.594 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:14 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-14T11:54:14.594 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:14 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-14T11:54:14.595 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:14 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:14.595 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:14 smithi097 ceph-mon[129313]: Deploying daemon mgr.y on smithi093 2023-07-14T11:54:14.595 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:14 smithi097 ceph-mon[129313]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:14.819 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:14 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:14.820 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:14 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:14.820 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:14 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:14.820 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:14 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-14T11:54:14.820 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:14 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-14T11:54:14.821 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:14 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-14T11:54:14.821 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:14 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:14.821 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:14 smithi093 ceph-mon[128922]: Deploying daemon mgr.y on smithi093 2023-07-14T11:54:14.821 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:14 smithi093 ceph-mon[128922]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:14.829 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:14 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:14.829 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:14 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:14.830 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:14 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:14.830 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:14 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-14T11:54:14.830 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:14 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-14T11:54:14.830 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:14 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-14T11:54:14.830 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:14 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:14.831 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:14 smithi017 ceph-mon[126339]: Deploying daemon mgr.y on smithi093 2023-07-14T11:54:14.831 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:14 smithi017 ceph-mon[126339]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:14.886 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Jul 14 11:54:14 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-z[131413]: 2023-07-14T11:54:14.609+0000 7fe76058e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-14T11:54:15.114 INFO:journalctl@ceph.mgr.y.smithi093.stdout:Jul 14 11:54:15 smithi093 systemd[1]: Starting Ceph mgr.y for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T11:54:15.386 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Jul 14 11:54:15 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-z[131413]: 2023-07-14T11:54:15.025+0000 7fe76058e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-14T11:54:15.574 INFO:journalctl@ceph.mgr.y.smithi093.stdout:Jul 14 11:54:15 smithi093 podman[130356]: 2023-07-14T11:54:15.620 INFO:teuthology.orchestra.run.smithi017.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-07-14T11:54:15.620 INFO:teuthology.orchestra.run.smithi017.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-07-14T11:54:15.621 INFO:teuthology.orchestra.run.smithi017.stderr: stderr: 10+0 records in 2023-07-14T11:54:15.621 INFO:teuthology.orchestra.run.smithi017.stderr:10+0 records out 2023-07-14T11:54:15.621 INFO:teuthology.orchestra.run.smithi017.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0487338 s, 215 MB/s 2023-07-14T11:54:15.621 INFO:teuthology.orchestra.run.smithi017.stderr:--> Zapping successful for: 2023-07-14T11:54:15.863 INFO:journalctl@ceph.mgr.y.smithi093.stdout:Jul 14 11:54:15 smithi093 bash[130356]: 4d7dc8d530ca15697ebee4287debb3035c949e4c273c6864ac5db7f983d4490b 2023-07-14T11:54:15.864 INFO:journalctl@ceph.mgr.y.smithi093.stdout:Jul 14 11:54:15 smithi093 ceph-mgr[130396]: -- 172.21.15.93:0/1503939440 <== mon.2 v2:172.21.15.93:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559592dc91e0 con 0x559592d9e800 2023-07-14T11:54:15.864 INFO:journalctl@ceph.mgr.y.smithi093.stdout:Jul 14 11:54:15 smithi093 systemd[1]: Started Ceph mgr.y for b3c386cc-223c-11ee-9b33-001a4aab830c. 2023-07-14T11:54:15.864 INFO:journalctl@ceph.mgr.y.smithi093.stdout:Jul 14 11:54:15 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-y[130372]: 2023-07-14T11:54:15.714+0000 7fc491a67200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-14T11:54:15.864 INFO:journalctl@ceph.mgr.y.smithi093.stdout:Jul 14 11:54:15 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-y[130372]: 2023-07-14T11:54:15.793+0000 7fc491a67200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-14T11:54:15.886 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Jul 14 11:54:15 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-z[131413]: 2023-07-14T11:54:15.591+0000 7fe76058e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-14T11:54:15.887 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Jul 14 11:54:15 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-z[131413]: 2023-07-14T11:54:15.760+0000 7fe76058e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-14T11:54:16.089 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi017:vg_nvme/lv_4 2023-07-14T11:54:16.145 INFO:journalctl@ceph.mgr.y.smithi093.stdout:Jul 14 11:54:15 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-y[130372]: 2023-07-14T11:54:15.886+0000 7fc491a67200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-14T11:54:16.145 INFO:journalctl@ceph.mgr.y.smithi093.stdout:Jul 14 11:54:15 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-y[130372]: 2023-07-14T11:54:15.984+0000 7fc491a67200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-14T11:54:16.459 INFO:journalctl@ceph.mgr.y.smithi093.stdout:Jul 14 11:54:16 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-y[130372]: 2023-07-14T11:54:16.163+0000 7fc491a67200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-14T11:54:16.779 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:16 smithi093 ceph-mon[128922]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:16.779 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:16 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:16.780 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:16 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:16.780 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:16 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:16.780 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:16 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:16.780 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:16 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:54:16.780 INFO:journalctl@ceph.mgr.y.smithi093.stdout:Jul 14 11:54:16 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-y[130372]: 2023-07-14T11:54:16.477+0000 7fc491a67200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-14T11:54:16.780 INFO:journalctl@ceph.mgr.y.smithi093.stdout:Jul 14 11:54:16 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-y[130372]: 2023-07-14T11:54:16.569+0000 7fc491a67200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-14T11:54:16.781 INFO:journalctl@ceph.mgr.y.smithi093.stdout:Jul 14 11:54:16 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-y[130372]: 2023-07-14T11:54:16.701+0000 7fc491a67200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-14T11:54:16.817 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T11:54:16.820 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Jul 14 11:54:16 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-z[131413]: 2023-07-14T11:54:16.566+0000 7fe76058e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-14T11:54:16.820 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Jul 14 11:54:16 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-z[131413]: 2023-07-14T11:54:16.662+0000 7fe76058e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-14T11:54:16.820 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:16 smithi097 ceph-mon[129313]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:16.821 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:16 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:16.821 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:16 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:16.821 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:16 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:16.821 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:16 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:16.821 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:16 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:54:17.113 INFO:journalctl@ceph.mgr.y.smithi093.stdout:Jul 14 11:54:16 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-y[130372]: 2023-07-14T11:54:16.797+0000 7fc491a67200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-14T11:54:17.116 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Jul 14 11:54:16 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-z[131413]: 2023-07-14T11:54:16.837+0000 7fe76058e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-14T11:54:17.118 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:16 smithi017 ceph-mon[126339]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:17.118 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:16 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:17.118 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:16 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:17.118 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:16 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:17.118 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:16 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:17.118 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:16 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:54:17.387 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Jul 14 11:54:17 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-z[131413]: 2023-07-14T11:54:17.132+0000 7fe76058e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-14T11:54:17.387 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Jul 14 11:54:17 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-z[131413]: 2023-07-14T11:54:17.366+0000 7fe76058e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-14T11:54:17.613 INFO:journalctl@ceph.mgr.y.smithi093.stdout:Jul 14 11:54:17 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-y[130372]: 2023-07-14T11:54:17.238+0000 7fc491a67200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-14T11:54:17.760 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Jul 14 11:54:17 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-z[131413]: 2023-07-14T11:54:17.517+0000 7fe76058e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-14T11:54:18.113 INFO:journalctl@ceph.mgr.y.smithi093.stdout:Jul 14 11:54:17 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-y[130372]: 2023-07-14T11:54:17.818+0000 7fc491a67200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-14T11:54:18.114 INFO:journalctl@ceph.mgr.y.smithi093.stdout:Jul 14 11:54:17 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-y[130372]: 2023-07-14T11:54:17.986+0000 7fc491a67200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-14T11:54:18.136 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Jul 14 11:54:17 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-z[131413]: 2023-07-14T11:54:17.777+0000 7fe76058e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-14T11:54:18.474 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:18.491+0000 7f5a30d7e700 1 -- 172.21.15.17:0/1699343690 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5a2c100d90 msgr2=0x7f5a2c1011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:18.475 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:18.491+0000 7f5a30d7e700 1 --2- 172.21.15.17:0/1699343690 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5a2c100d90 0x7f5a2c1011e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5a200098f0 tx=0x7f5a20009c00 comp rx=0 tx=0).stop 2023-07-14T11:54:18.475 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:18.491+0000 7f5a30d7e700 1 -- 172.21.15.17:0/1699343690 shutdown_connections 2023-07-14T11:54:18.475 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:18.491+0000 7f5a30d7e700 1 --2- 172.21.15.17:0/1699343690 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a2c101720 0x7f5a2c107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:18.475 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:18.491+0000 7f5a30d7e700 1 --2- 172.21.15.17:0/1699343690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a2c0ffb90 0x7f5a2c0fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:18.475 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:18.491+0000 7f5a30d7e700 1 --2- 172.21.15.17:0/1699343690 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5a2c100d90 0x7f5a2c1011e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:18.476 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:18.491+0000 7f5a30d7e700 1 -- 172.21.15.17:0/1699343690 >> 172.21.15.17:0/1699343690 conn(0x7f5a2c0fb1a0 msgr2=0x7f5a2c0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:18.476 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:18.491+0000 7f5a30d7e700 1 -- 172.21.15.17:0/1699343690 shutdown_connections 2023-07-14T11:54:18.476 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:18.491+0000 7f5a30d7e700 1 -- 172.21.15.17:0/1699343690 wait complete. 2023-07-14T11:54:18.478 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:18.492+0000 7f5a30d7e700 1 Processor -- start 2023-07-14T11:54:18.478 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:18.492+0000 7f5a30d7e700 1 -- start start 2023-07-14T11:54:18.478 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:18.492+0000 7f5a30d7e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a2c0ffb90 0x7f5a2c197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:18.479 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:18.492+0000 7f5a30d7e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a2c100d90 0x7f5a2c198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:18.479 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:18.492+0000 7f5a30d7e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5a2c101720 0x7f5a2c19f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:18.479 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:18.492+0000 7f5a30d7e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2c19f6e0 con 0x7f5a2c100d90 2023-07-14T11:54:18.480 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:18.492+0000 7f5a30d7e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2c19f820 con 0x7f5a2c101720 2023-07-14T11:54:18.480 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:18.492+0000 7f5a30d7e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2c19f9b0 con 0x7f5a2c0ffb90 2023-07-14T11:54:18.480 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:18.493+0000 7f5a2a59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a2c0ffb90 0x7f5a2c197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:18.481 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:18.493+0000 7f5a2a59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a2c0ffb90 0x7f5a2c197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.17:46478/0 (socket says 172.21.15.17:46478) 2023-07-14T11:54:18.481 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:18.493+0000 7f5a2a59c700 1 -- 172.21.15.17:0/4061613184 learned_addr learned my addr 172.21.15.17:0/4061613184 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:54:18.481 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:18.493+0000 7f5a2ad9d700 1 --2- 172.21.15.17:0/4061613184 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5a2c101720 0x7f5a2c19f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:18.482 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:18.493+0000 7f5a29d9b700 1 --2- 172.21.15.17:0/4061613184 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a2c100d90 0x7f5a2c198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:18.482 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:18.493+0000 7f5a2a59c700 1 -- 172.21.15.17:0/4061613184 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5a2c101720 msgr2=0x7f5a2c19f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:18.482 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:18.493+0000 7f5a2a59c700 1 --2- 172.21.15.17:0/4061613184 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5a2c101720 0x7f5a2c19f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:18.482 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:18.493+0000 7f5a2a59c700 1 -- 172.21.15.17:0/4061613184 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a2c100d90 msgr2=0x7f5a2c198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:18.483 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:18.493+0000 7f5a2a59c700 1 --2- 172.21.15.17:0/4061613184 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a2c100d90 0x7f5a2c198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:18.483 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:18.493+0000 7f5a2a59c700 1 -- 172.21.15.17:0/4061613184 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a20009520 con 0x7f5a2c0ffb90 2023-07-14T11:54:18.483 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:18.493+0000 7f5a2a59c700 1 --2- 172.21.15.17:0/4061613184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a2c0ffb90 0x7f5a2c197b50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c00ea20 tx=0x7f5a1c00ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:18.483 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:18.494+0000 7f5a1b7fe700 1 -- 172.21.15.17:0/4061613184 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c004220 con 0x7f5a2c0ffb90 2023-07-14T11:54:18.484 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:18.494+0000 7f5a1b7fe700 1 -- 172.21.15.17:0/4061613184 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a1c004380 con 0x7f5a2c0ffb90 2023-07-14T11:54:18.484 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:18.494+0000 7f5a30d7e700 1 -- 172.21.15.17:0/4061613184 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a2c19fc90 con 0x7f5a2c0ffb90 2023-07-14T11:54:18.484 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:18.494+0000 7f5a1b7fe700 1 -- 172.21.15.17:0/4061613184 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c003be0 con 0x7f5a2c0ffb90 2023-07-14T11:54:18.485 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:18.494+0000 7f5a30d7e700 1 -- 172.21.15.17:0/4061613184 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a2c1a0190 con 0x7f5a2c0ffb90 2023-07-14T11:54:18.485 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:18.495+0000 7f5a30d7e700 1 -- 172.21.15.17:0/4061613184 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a2c192260 con 0x7f5a2c0ffb90 2023-07-14T11:54:18.485 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:18.495+0000 7f5a1b7fe700 1 -- 172.21.15.17:0/4061613184 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f5a1c003d40 con 0x7f5a2c0ffb90 2023-07-14T11:54:18.486 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:18.495+0000 7f5a1b7fe700 1 --2- 172.21.15.17:0/4061613184 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f5a140389e0 0x7f5a1403ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:18.486 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:18.495+0000 7f5a1b7fe700 1 -- 172.21.15.17:0/4061613184 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5a1c014070 con 0x7f5a2c0ffb90 2023-07-14T11:54:18.486 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:18.495+0000 7f5a29d9b700 1 --2- 172.21.15.17:0/4061613184 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f5a140389e0 0x7f5a1403ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:18.487 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:18.496+0000 7f5a29d9b700 1 --2- 172.21.15.17:0/4061613184 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f5a140389e0 0x7f5a1403ae90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5a20005e90 tx=0x7f5a20019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:18.487 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:18.504+0000 7f5a1b7fe700 1 -- 172.21.15.17:0/4061613184 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5a1c01f520 con 0x7f5a2c0ffb90 2023-07-14T11:54:18.598 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:18 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:18.599 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:18 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:18.599 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:18 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:18.599 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:18 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:18.599 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:18 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:18.599 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:18 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:54:18.599 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:18 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:18.599 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:18 smithi017 ceph-mon[126339]: Reconfiguring mgr.x (unknown last config time)... 2023-07-14T11:54:18.600 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:18 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-14T11:54:18.600 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:18 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-14T11:54:18.600 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:18 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:18.600 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:18 smithi017 ceph-mon[126339]: Reconfiguring daemon mgr.x on smithi017 2023-07-14T11:54:18.600 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:18 smithi017 ceph-mon[126339]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:18.713 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:18.730+0000 7f5a30d7e700 1 -- 172.21.15.17:0/4061613184 --> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a2c060950 con 0x7f5a140389e0 2023-07-14T11:54:18.720 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:18 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:18.720 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:18 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:18.720 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:18 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:18.720 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:18 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:18.721 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:18 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:18.721 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:18 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:54:18.721 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:18 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:18.722 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:18 smithi093 ceph-mon[128922]: Reconfiguring mgr.x (unknown last config time)... 2023-07-14T11:54:18.722 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:18 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-14T11:54:18.722 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:18 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-14T11:54:18.722 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:18 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:18.723 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:18 smithi093 ceph-mon[128922]: Reconfiguring daemon mgr.x on smithi017 2023-07-14T11:54:18.723 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:18 smithi093 ceph-mon[128922]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:18.773 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Jul 14 11:54:18 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-z[131413]: 2023-07-14T11:54:18.468+0000 7fe76058e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-14T11:54:18.774 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Jul 14 11:54:18 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-z[131413]: 2023-07-14T11:54:18.603+0000 7fe76058e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-14T11:54:18.774 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Jul 14 11:54:18 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-z[131413]: 2023-07-14T11:54:18.790+0000 7fe76058e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-14T11:54:18.774 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:18 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:18.774 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:18 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:18.774 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:18 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:18.775 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:18 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:18.775 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:18 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:18.775 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:18 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:54:18.775 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:18 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:18.775 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:18 smithi097 ceph-mon[129313]: Reconfiguring mgr.x (unknown last config time)... 2023-07-14T11:54:18.775 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:18 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-14T11:54:18.775 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:18 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-14T11:54:18.775 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:18 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:18.776 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:18 smithi097 ceph-mon[129313]: Reconfiguring daemon mgr.x on smithi017 2023-07-14T11:54:18.776 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:18 smithi097 ceph-mon[129313]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:18.977 INFO:journalctl@ceph.mgr.y.smithi093.stdout:Jul 14 11:54:18 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-y[130372]: 2023-07-14T11:54:18.737+0000 7fc491a67200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-14T11:54:18.977 INFO:journalctl@ceph.mgr.y.smithi093.stdout:Jul 14 11:54:18 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-y[130372]: 2023-07-14T11:54:18.827+0000 7fc491a67200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-14T11:54:18.977 INFO:journalctl@ceph.mgr.y.smithi093.stdout:Jul 14 11:54:18 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-y[130372]: 2023-07-14T11:54:18.995+0000 7fc491a67200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-14T11:54:19.096 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Jul 14 11:54:18 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-z[131413]: 2023-07-14T11:54:18.902+0000 7fe76058e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-14T11:54:19.386 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Jul 14 11:54:19 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-z[131413]: 2023-07-14T11:54:19.113+0000 7fe76058e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-14T11:54:19.613 INFO:journalctl@ceph.mgr.y.smithi093.stdout:Jul 14 11:54:19 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-y[130372]: 2023-07-14T11:54:19.268+0000 7fc491a67200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-14T11:54:19.614 INFO:journalctl@ceph.mgr.y.smithi093.stdout:Jul 14 11:54:19 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-y[130372]: 2023-07-14T11:54:19.491+0000 7fc491a67200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-14T11:54:19.614 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:19 smithi093 ceph-mon[128922]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:54:19.614 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:19 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:54:19.615 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:19 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:54:19.615 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:19 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:19.615 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:19 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:19.615 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:19 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:19.615 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:19 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:54:19.616 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:19 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:19.616 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:19 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:54:19.616 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:19 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:19.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:19 smithi017 ceph-mon[126339]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:54:19.647 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:19 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:54:19.647 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:19 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:54:19.647 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:19 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:19.647 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:19 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:19.647 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:19 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:19.647 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:19 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:54:19.648 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:19 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:19.648 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:19 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:54:19.648 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:19 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:19.881 INFO:journalctl@ceph.mgr.y.smithi093.stdout:Jul 14 11:54:19 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-y[130372]: 2023-07-14T11:54:19.643+0000 7fc491a67200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-14T11:54:19.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:19 smithi097 ceph-mon[129313]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:54:19.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:19 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:54:19.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:19 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:54:19.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:19 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:19.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:19 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:19.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:19 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:19.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:19 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:54:19.888 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:19 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:19.888 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:19 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:54:19.888 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:19 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:20.363 INFO:journalctl@ceph.mgr.y.smithi093.stdout:Jul 14 11:54:19 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-y[130372]: 2023-07-14T11:54:19.899+0000 7fc491a67200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-14T11:54:20.387 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Jul 14 11:54:19 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-z[131413]: 2023-07-14T11:54:19.961+0000 7fe76058e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-14T11:54:20.458 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:20.475+0000 7f5a1b7fe700 1 -- 172.21.15.17:0/4061613184 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mgrmap(e 14) v1 ==== 90599+0+0 (secure 0 0 0) 0x7f5a1c01dd20 con 0x7f5a2c0ffb90 2023-07-14T11:54:20.838 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:20 smithi017 ceph-mon[126339]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:20.839 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:20 smithi017 ceph-mon[126339]: Standby manager daemon z started 2023-07-14T11:54:20.839 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:20 smithi017 ceph-mon[126339]: from='mgr.? 172.21.15.97:0/3223749117' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-14T11:54:20.839 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:20 smithi017 ceph-mon[126339]: from='mgr.? 172.21.15.97:0/3223749117' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-14T11:54:20.839 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:20 smithi017 ceph-mon[126339]: from='mgr.? 172.21.15.97:0/3223749117' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-14T11:54:20.839 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:20 smithi017 ceph-mon[126339]: from='mgr.? 172.21.15.97:0/3223749117' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-14T11:54:20.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:20 smithi093 ceph-mon[128922]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:20.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:20 smithi093 ceph-mon[128922]: Standby manager daemon z started 2023-07-14T11:54:20.864 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:20 smithi093 ceph-mon[128922]: from='mgr.? 172.21.15.97:0/3223749117' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-14T11:54:20.864 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:20 smithi093 ceph-mon[128922]: from='mgr.? 172.21.15.97:0/3223749117' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-14T11:54:20.864 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:20 smithi093 ceph-mon[128922]: from='mgr.? 172.21.15.97:0/3223749117' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-14T11:54:20.864 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:20 smithi093 ceph-mon[128922]: from='mgr.? 172.21.15.97:0/3223749117' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-14T11:54:20.864 INFO:journalctl@ceph.mgr.y.smithi093.stdout:Jul 14 11:54:20 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-y[130372]: 2023-07-14T11:54:20.589+0000 7fc491a67200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-14T11:54:20.865 INFO:journalctl@ceph.mgr.y.smithi093.stdout:Jul 14 11:54:20 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-y[130372]: 2023-07-14T11:54:20.718+0000 7fc491a67200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-14T11:54:20.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:20 smithi097 ceph-mon[129313]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:20.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:20 smithi097 ceph-mon[129313]: Standby manager daemon z started 2023-07-14T11:54:20.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:20 smithi097 ceph-mon[129313]: from='mgr.? 172.21.15.97:0/3223749117' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-14T11:54:20.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:20 smithi097 ceph-mon[129313]: from='mgr.? 172.21.15.97:0/3223749117' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-14T11:54:20.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:20 smithi097 ceph-mon[129313]: from='mgr.? 172.21.15.97:0/3223749117' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-14T11:54:20.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:20 smithi097 ceph-mon[129313]: from='mgr.? 172.21.15.97:0/3223749117' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-14T11:54:21.214 INFO:journalctl@ceph.mgr.y.smithi093.stdout:Jul 14 11:54:20 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-y[130372]: 2023-07-14T11:54:20.906+0000 7fc491a67200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-14T11:54:21.214 INFO:journalctl@ceph.mgr.y.smithi093.stdout:Jul 14 11:54:21 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-y[130372]: 2023-07-14T11:54:21.019+0000 7fc491a67200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-14T11:54:21.214 INFO:journalctl@ceph.mgr.y.smithi093.stdout:Jul 14 11:54:21 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-y[130372]: 2023-07-14T11:54:21.232+0000 7fc491a67200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-14T11:54:21.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:21 smithi093 ceph-mon[128922]: mgrmap e14: x(active, since 74s), standbys: z 2023-07-14T11:54:21.614 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:21 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-14T11:54:21.614 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:21 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:21.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:21 smithi097 ceph-mon[129313]: mgrmap e14: x(active, since 74s), standbys: z 2023-07-14T11:54:21.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:21 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-14T11:54:21.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:21 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:21.896 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:21 smithi017 ceph-mon[126339]: mgrmap e14: x(active, since 74s), standbys: z 2023-07-14T11:54:21.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:21 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-14T11:54:21.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:21 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:22.363 INFO:journalctl@ceph.mgr.y.smithi093.stdout:Jul 14 11:54:22 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-y[130372]: 2023-07-14T11:54:22.077+0000 7fc491a67200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-14T11:54:22.855 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:22.872+0000 7f5a1b7fe700 1 -- 172.21.15.17:0/4061613184 <== mon.1 v2:172.21.15.97:3300/0 8 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7f5a1c020c70 con 0x7f5a2c0ffb90 2023-07-14T11:54:22.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:22 smithi093 ceph-mon[128922]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:22.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:22 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/4127222404' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57003438-a83c-4c8e-a029-56414740895c"}]: dispatch 2023-07-14T11:54:22.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:22 smithi093 ceph-mon[128922]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57003438-a83c-4c8e-a029-56414740895c"}]: dispatch 2023-07-14T11:54:22.864 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:22 smithi093 ceph-mon[128922]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "57003438-a83c-4c8e-a029-56414740895c"}]': finished 2023-07-14T11:54:22.864 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:22 smithi093 ceph-mon[128922]: osdmap e5: 1 total, 0 up, 1 in 2023-07-14T11:54:22.864 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:22 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T11:54:22.864 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:22 smithi093 ceph-mon[128922]: Standby manager daemon y started 2023-07-14T11:54:22.864 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:22 smithi093 ceph-mon[128922]: from='mgr.? 172.21.15.93:0/3316881553' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-14T11:54:22.864 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:22 smithi093 ceph-mon[128922]: from='mgr.? 172.21.15.93:0/3316881553' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-14T11:54:22.865 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:22 smithi093 ceph-mon[128922]: from='mgr.? 172.21.15.93:0/3316881553' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-14T11:54:22.865 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:22 smithi093 ceph-mon[128922]: from='mgr.? 172.21.15.93:0/3316881553' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-14T11:54:22.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:22 smithi097 ceph-mon[129313]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:22.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:22 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/4127222404' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57003438-a83c-4c8e-a029-56414740895c"}]: dispatch 2023-07-14T11:54:22.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:22 smithi097 ceph-mon[129313]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57003438-a83c-4c8e-a029-56414740895c"}]: dispatch 2023-07-14T11:54:22.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:22 smithi097 ceph-mon[129313]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "57003438-a83c-4c8e-a029-56414740895c"}]': finished 2023-07-14T11:54:22.888 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:22 smithi097 ceph-mon[129313]: osdmap e5: 1 total, 0 up, 1 in 2023-07-14T11:54:22.888 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:22 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T11:54:22.888 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:22 smithi097 ceph-mon[129313]: Standby manager daemon y started 2023-07-14T11:54:22.888 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:22 smithi097 ceph-mon[129313]: from='mgr.? 172.21.15.93:0/3316881553' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-14T11:54:22.888 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:22 smithi097 ceph-mon[129313]: from='mgr.? 172.21.15.93:0/3316881553' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-14T11:54:22.888 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:22 smithi097 ceph-mon[129313]: from='mgr.? 172.21.15.93:0/3316881553' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-14T11:54:22.888 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:22 smithi097 ceph-mon[129313]: from='mgr.? 172.21.15.93:0/3316881553' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-14T11:54:22.896 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:22 smithi017 ceph-mon[126339]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:22.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:22 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/4127222404' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57003438-a83c-4c8e-a029-56414740895c"}]: dispatch 2023-07-14T11:54:22.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:22 smithi017 ceph-mon[126339]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57003438-a83c-4c8e-a029-56414740895c"}]: dispatch 2023-07-14T11:54:22.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:22 smithi017 ceph-mon[126339]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "57003438-a83c-4c8e-a029-56414740895c"}]': finished 2023-07-14T11:54:22.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:22 smithi017 ceph-mon[126339]: osdmap e5: 1 total, 0 up, 1 in 2023-07-14T11:54:22.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:22 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T11:54:22.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:22 smithi017 ceph-mon[126339]: Standby manager daemon y started 2023-07-14T11:54:22.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:22 smithi017 ceph-mon[126339]: from='mgr.? 172.21.15.93:0/3316881553' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-14T11:54:22.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:22 smithi017 ceph-mon[126339]: from='mgr.? 172.21.15.93:0/3316881553' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-14T11:54:22.898 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:22 smithi017 ceph-mon[126339]: from='mgr.? 172.21.15.93:0/3316881553' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-14T11:54:22.898 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:22 smithi017 ceph-mon[126339]: from='mgr.? 172.21.15.93:0/3316881553' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-14T11:54:23.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:23 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/1398146309' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:54:23.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:23 smithi093 ceph-mon[128922]: mgrmap e15: x(active, since 77s), standbys: z, y 2023-07-14T11:54:23.864 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:23 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-14T11:54:23.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:23 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/1398146309' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:54:23.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:23 smithi097 ceph-mon[129313]: mgrmap e15: x(active, since 77s), standbys: z, y 2023-07-14T11:54:23.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:23 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-14T11:54:23.896 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:23 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/1398146309' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:54:23.896 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:23 smithi017 ceph-mon[126339]: mgrmap e15: x(active, since 77s), standbys: z, y 2023-07-14T11:54:23.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:23 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-14T11:54:24.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:24 smithi093 ceph-mon[128922]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:24.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:24 smithi097 ceph-mon[129313]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:24.896 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:24 smithi017 ceph-mon[126339]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:27.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:26 smithi093 ceph-mon[128922]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:27.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:26 smithi097 ceph-mon[129313]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:27.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:26 smithi017 ceph-mon[126339]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:29.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:28 smithi097 ceph-mon[129313]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:29.137 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:28 smithi017 ceph-mon[126339]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:29.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:28 smithi093 ceph-mon[128922]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:30.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:29 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-07-14T11:54:30.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:29 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:30.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:29 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-07-14T11:54:30.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:29 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:30.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:29 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-07-14T11:54:30.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:29 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:31.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:30 smithi017 ceph-mon[126339]: Deploying daemon osd.0 on smithi017 2023-07-14T11:54:31.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:30 smithi017 ceph-mon[126339]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:31.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:30 smithi093 ceph-mon[128922]: Deploying daemon osd.0 on smithi017 2023-07-14T11:54:31.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:30 smithi093 ceph-mon[128922]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:31.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:30 smithi097 ceph-mon[129313]: Deploying daemon osd.0 on smithi017 2023-07-14T11:54:31.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:30 smithi097 ceph-mon[129313]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:33.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:32 smithi017 ceph-mon[126339]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:33.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:32 smithi093 ceph-mon[128922]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:33.388 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:32 smithi097 ceph-mon[129313]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:35.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:34 smithi017 ceph-mon[126339]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:35.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:34 smithi093 ceph-mon[128922]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:35.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:34 smithi097 ceph-mon[129313]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:37.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:36 smithi017 ceph-mon[126339]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:37.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:36 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:37.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:36 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:37.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:36 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:54:37.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:36 smithi093 ceph-mon[128922]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:37.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:36 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:37.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:36 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:37.364 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:36 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:54:37.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:36 smithi097 ceph-mon[129313]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:37.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:36 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:37.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:36 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:37.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:36 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:54:39.241 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:38 smithi017 ceph-mon[126339]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:39.241 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:38 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:39.241 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:38 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:39.242 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:38 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:39.242 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:38 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:54:39.242 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:38 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:39.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:38 smithi093 ceph-mon[128922]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:39.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:38 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:39.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:38 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:39.364 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:38 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:39.364 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:38 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:54:39.364 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:38 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:39.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:38 smithi097 ceph-mon[129313]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:39.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:38 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:39.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:38 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:39.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:38 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:39.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:38 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:54:39.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:38 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:39.725 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:39.742+0000 7f5a1b7fe700 1 -- 172.21.15.17:0/4061613184 <== mgr.14152 v2:172.21.15.17:6800/4237449858 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5a2c060950 con 0x7f5a140389e0 2023-07-14T11:54:39.726 INFO:teuthology.orchestra.run.smithi017.stdout:Created osd(s) 0 on host 'smithi017' 2023-07-14T11:54:39.728 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:39.745+0000 7f5a30d7e700 1 -- 172.21.15.17:0/4061613184 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f5a140389e0 msgr2=0x7f5a1403ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:39.729 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:39.745+0000 7f5a30d7e700 1 --2- 172.21.15.17:0/4061613184 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f5a140389e0 0x7f5a1403ae90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5a20005e90 tx=0x7f5a20019040 comp rx=0 tx=0).stop 2023-07-14T11:54:39.729 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:39.745+0000 7f5a30d7e700 1 -- 172.21.15.17:0/4061613184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a2c0ffb90 msgr2=0x7f5a2c197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:39.729 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:39.745+0000 7f5a30d7e700 1 --2- 172.21.15.17:0/4061613184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a2c0ffb90 0x7f5a2c197b50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c00ea20 tx=0x7f5a1c00ed30 comp rx=0 tx=0).stop 2023-07-14T11:54:39.729 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:39.746+0000 7f5a30d7e700 1 -- 172.21.15.17:0/4061613184 shutdown_connections 2023-07-14T11:54:39.729 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:39.746+0000 7f5a30d7e700 1 --2- 172.21.15.17:0/4061613184 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f5a140389e0 0x7f5a1403ae90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:39.729 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:39.746+0000 7f5a30d7e700 1 --2- 172.21.15.17:0/4061613184 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5a2c101720 0x7f5a2c19f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:39.729 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:39.746+0000 7f5a30d7e700 1 --2- 172.21.15.17:0/4061613184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a2c0ffb90 0x7f5a2c197b50 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:39.730 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:39.746+0000 7f5a30d7e700 1 --2- 172.21.15.17:0/4061613184 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a2c100d90 0x7f5a2c198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:39.730 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:39.746+0000 7f5a30d7e700 1 -- 172.21.15.17:0/4061613184 >> 172.21.15.17:0/4061613184 conn(0x7f5a2c0fb1a0 msgr2=0x7f5a2c106760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:39.730 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:39.746+0000 7f5a30d7e700 1 -- 172.21.15.17:0/4061613184 shutdown_connections 2023-07-14T11:54:39.730 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:54:39.746+0000 7f5a30d7e700 1 -- 172.21.15.17:0/4061613184 wait complete. 2023-07-14T11:54:40.251 DEBUG:teuthology.orchestra.run.smithi017:osd.0> sudo journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@osd.0.service 2023-07-14T11:54:40.256 INFO:tasks.cephadm:Deploying osd.1 on smithi093 with /dev/vg_nvme/lv_4... 2023-07-14T11:54:40.256 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-07-14T11:54:40.337 INFO:journalctl@ceph.osd.0.smithi017.stdout:-- Logs begin at Fri 2023-07-14 11:32:03 UTC. -- 2023-07-14T11:54:40.818 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.b/config 2023-07-14T11:54:40.959 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:40 smithi017 ceph-mon[126339]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:40.960 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:40 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:40.960 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:40 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:40.960 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:40 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:54:40.960 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:40 smithi017 ceph-mon[126339]: from='osd.0 [v2:172.21.15.17:6802/3588366384,v1:172.21.15.17:6803/3588366384]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-14T11:54:40.961 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:40 smithi017 ceph-mon[126339]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-14T11:54:41.027 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:40 smithi093 ceph-mon[128922]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:41.027 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:40 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:41.028 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:40 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:41.028 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:40 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:54:41.028 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:40 smithi093 ceph-mon[128922]: from='osd.0 [v2:172.21.15.17:6802/3588366384,v1:172.21.15.17:6803/3588366384]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-14T11:54:41.028 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:40 smithi093 ceph-mon[128922]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-14T11:54:41.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:40 smithi097 ceph-mon[129313]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:41.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:40 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:41.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:40 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:41.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:40 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:54:41.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:40 smithi097 ceph-mon[129313]: from='osd.0 [v2:172.21.15.17:6802/3588366384,v1:172.21.15.17:6803/3588366384]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-14T11:54:41.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:40 smithi097 ceph-mon[129313]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-14T11:54:42.011 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:41 smithi093 ceph-mon[128922]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-07-14T11:54:42.011 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:41 smithi093 ceph-mon[128922]: osdmap e6: 1 total, 0 up, 1 in 2023-07-14T11:54:42.011 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:41 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T11:54:42.011 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:41 smithi093 ceph-mon[128922]: from='osd.0 [v2:172.21.15.17:6802/3588366384,v1:172.21.15.17:6803/3588366384]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2023-07-14T11:54:42.011 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:41 smithi093 ceph-mon[128922]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2023-07-14T11:54:42.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:41 smithi017 ceph-mon[126339]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-07-14T11:54:42.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:41 smithi017 ceph-mon[126339]: osdmap e6: 1 total, 0 up, 1 in 2023-07-14T11:54:42.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:41 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T11:54:42.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:41 smithi017 ceph-mon[126339]: from='osd.0 [v2:172.21.15.17:6802/3588366384,v1:172.21.15.17:6803/3588366384]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2023-07-14T11:54:42.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:41 smithi017 ceph-mon[126339]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2023-07-14T11:54:42.148 INFO:journalctl@ceph.osd.0.smithi017.stdout:Jul 14 11:54:41 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-0[139449]: 2023-07-14T11:54:41.931+0000 7fdce1648700 -1 osd.0 0 waiting for initial osdmap 2023-07-14T11:54:42.148 INFO:journalctl@ceph.osd.0.smithi017.stdout:Jul 14 11:54:41 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-0[139449]: 2023-07-14T11:54:41.935+0000 7fdcdac38700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T11:54:42.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:41 smithi097 ceph-mon[129313]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-07-14T11:54:42.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:41 smithi097 ceph-mon[129313]: osdmap e6: 1 total, 0 up, 1 in 2023-07-14T11:54:42.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:41 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T11:54:42.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:41 smithi097 ceph-mon[129313]: from='osd.0 [v2:172.21.15.17:6802/3588366384,v1:172.21.15.17:6803/3588366384]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2023-07-14T11:54:42.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:41 smithi097 ceph-mon[129313]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2023-07-14T11:54:42.492 INFO:teuthology.orchestra.run.smithi093.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-07-14T11:54:42.492 INFO:teuthology.orchestra.run.smithi093.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-07-14T11:54:42.492 INFO:teuthology.orchestra.run.smithi093.stderr: stderr: 10+0 records in 2023-07-14T11:54:42.492 INFO:teuthology.orchestra.run.smithi093.stderr:10+0 records out 2023-07-14T11:54:42.492 INFO:teuthology.orchestra.run.smithi093.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0143895 s, 729 MB/s 2023-07-14T11:54:42.493 INFO:teuthology.orchestra.run.smithi093.stderr:--> Zapping successful for: 2023-07-14T11:54:42.857 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi093:vg_nvme/lv_4 2023-07-14T11:54:43.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:42 smithi093 ceph-mon[128922]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:43.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:42 smithi093 ceph-mon[128922]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2023-07-14T11:54:43.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:42 smithi093 ceph-mon[128922]: osdmap e7: 1 total, 0 up, 1 in 2023-07-14T11:54:43.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:42 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T11:54:43.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:42 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T11:54:43.184 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:42 smithi017 ceph-mon[126339]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:43.185 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:42 smithi017 ceph-mon[126339]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2023-07-14T11:54:43.185 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:42 smithi017 ceph-mon[126339]: osdmap e7: 1 total, 0 up, 1 in 2023-07-14T11:54:43.185 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:42 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T11:54:43.185 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:42 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T11:54:43.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:42 smithi097 ceph-mon[129313]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T11:54:43.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:42 smithi097 ceph-mon[129313]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2023-07-14T11:54:43.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:42 smithi097 ceph-mon[129313]: osdmap e7: 1 total, 0 up, 1 in 2023-07-14T11:54:43.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:42 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T11:54:43.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:42 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T11:54:43.411 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.b/config 2023-07-14T11:54:44.298 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:43 smithi093 ceph-mon[128922]: purged_snaps scrub starts 2023-07-14T11:54:44.298 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:43 smithi093 ceph-mon[128922]: purged_snaps scrub ok 2023-07-14T11:54:44.299 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:43 smithi093 ceph-mon[128922]: osd.0 [v2:172.21.15.17:6802/3588366384,v1:172.21.15.17:6803/3588366384] boot 2023-07-14T11:54:44.299 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:43 smithi093 ceph-mon[128922]: osdmap e8: 1 total, 1 up, 1 in 2023-07-14T11:54:44.299 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:43 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T11:54:44.299 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:43 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:44.299 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:43 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:44.300 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:43 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:54:44.300 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:43 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:44.300 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:43 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:44.301 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:43 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:54:44.301 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:43 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:44.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:43 smithi097 ceph-mon[129313]: purged_snaps scrub starts 2023-07-14T11:54:44.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:43 smithi097 ceph-mon[129313]: purged_snaps scrub ok 2023-07-14T11:54:44.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:43 smithi097 ceph-mon[129313]: osd.0 [v2:172.21.15.17:6802/3588366384,v1:172.21.15.17:6803/3588366384] boot 2023-07-14T11:54:44.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:43 smithi097 ceph-mon[129313]: osdmap e8: 1 total, 1 up, 1 in 2023-07-14T11:54:44.388 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:43 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T11:54:44.388 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:43 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:44.388 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:43 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:44.388 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:43 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:54:44.388 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:43 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:44.388 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:43 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:44.389 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:43 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:54:44.389 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:43 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:44.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:43 smithi017 ceph-mon[126339]: purged_snaps scrub starts 2023-07-14T11:54:44.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:43 smithi017 ceph-mon[126339]: purged_snaps scrub ok 2023-07-14T11:54:44.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:43 smithi017 ceph-mon[126339]: osd.0 [v2:172.21.15.17:6802/3588366384,v1:172.21.15.17:6803/3588366384] boot 2023-07-14T11:54:44.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:43 smithi017 ceph-mon[126339]: osdmap e8: 1 total, 1 up, 1 in 2023-07-14T11:54:44.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:43 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T11:54:44.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:43 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:44.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:43 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:44.399 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:43 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:54:44.399 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:43 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:44.399 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:43 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:44.400 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:43 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:54:44.400 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:43 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:44.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:54:44.699+0000 7feea8f29700 1 -- 172.21.15.93:0/2530345898 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feea4100610 msgr2=0x7feea4106ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:44.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:54:44.699+0000 7feea8f29700 1 --2- 172.21.15.93:0/2530345898 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feea4100610 0x7feea4106ed0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fee9c00b0a0 tx=0x7fee9c00b3b0 comp rx=0 tx=0).stop 2023-07-14T11:54:44.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:54:44.699+0000 7feea8f29700 1 -- 172.21.15.93:0/2530345898 shutdown_connections 2023-07-14T11:54:44.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:54:44.699+0000 7feea8f29700 1 --2- 172.21.15.93:0/2530345898 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feea4100610 0x7feea4106ed0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:44.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:54:44.699+0000 7feea8f29700 1 --2- 172.21.15.93:0/2530345898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feea40fea80 0x7feea40fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:44.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:54:44.699+0000 7feea8f29700 1 --2- 172.21.15.93:0/2530345898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feea40ffc80 0x7feea41000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:44.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:54:44.699+0000 7feea8f29700 1 -- 172.21.15.93:0/2530345898 >> 172.21.15.93:0/2530345898 conn(0x7feea40fa0b0 msgr2=0x7feea40fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:54:44.684 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:54:44.699+0000 7feea8f29700 1 -- 172.21.15.93:0/2530345898 shutdown_connections 2023-07-14T11:54:44.684 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:54:44.699+0000 7feea8f29700 1 -- 172.21.15.93:0/2530345898 wait complete. 2023-07-14T11:54:44.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:54:44.700+0000 7feea8f29700 1 Processor -- start 2023-07-14T11:54:44.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:54:44.700+0000 7feea8f29700 1 -- start start 2023-07-14T11:54:44.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:54:44.700+0000 7feea8f29700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feea40fea80 0x7feea41947b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:44.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:54:44.700+0000 7feea8f29700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feea40ffc80 0x7feea4194cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:44.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:54:44.700+0000 7feea8f29700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feea4100610 0x7feea419bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:44.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:54:44.700+0000 7feea8f29700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea419c340 con 0x7feea40ffc80 2023-07-14T11:54:44.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:54:44.700+0000 7feea8f29700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea419c480 con 0x7feea40fea80 2023-07-14T11:54:44.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:54:44.700+0000 7feea8f29700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea419c610 con 0x7feea4100610 2023-07-14T11:54:44.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:54:44.700+0000 7feea259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feea40fea80 0x7feea41947b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:44.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:54:44.700+0000 7feea259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feea40fea80 0x7feea41947b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:57596/0 (socket says 172.21.15.93:57596) 2023-07-14T11:54:44.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:54:44.700+0000 7feea259c700 1 -- 172.21.15.93:0/550517567 learned_addr learned my addr 172.21.15.93:0/550517567 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-14T11:54:44.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:54:44.701+0000 7feea259c700 1 -- 172.21.15.93:0/550517567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feea4100610 msgr2=0x7feea419bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:44.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:54:44.701+0000 7feea259c700 1 --2- 172.21.15.93:0/550517567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feea4100610 0x7feea419bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:44.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:54:44.701+0000 7feea259c700 1 -- 172.21.15.93:0/550517567 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feea40ffc80 msgr2=0x7feea4194cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:54:44.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:54:44.701+0000 7feea259c700 1 --2- 172.21.15.93:0/550517567 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feea40ffc80 0x7feea4194cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:54:44.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:54:44.701+0000 7feea259c700 1 -- 172.21.15.93:0/550517567 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee9c009d00 con 0x7feea40fea80 2023-07-14T11:54:44.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:54:44.701+0000 7feea259c700 1 --2- 172.21.15.93:0/550517567 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feea40fea80 0x7feea41947b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fee98007b50 tx=0x7fee98007f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:44.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:54:44.701+0000 7fee937fe700 1 -- 172.21.15.93:0/550517567 <== mon.2 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee98011070 con 0x7feea40fea80 2023-07-14T11:54:44.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:54:44.701+0000 7fee937fe700 1 -- 172.21.15.93:0/550517567 <== mon.2 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee9800c2e0 con 0x7feea40fea80 2023-07-14T11:54:44.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:54:44.701+0000 7fee937fe700 1 -- 172.21.15.93:0/550517567 <== mon.2 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9800e5c0 con 0x7feea40fea80 2023-07-14T11:54:44.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:54:44.701+0000 7feea8f29700 1 -- 172.21.15.93:0/550517567 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feea4068030 con 0x7feea40fea80 2023-07-14T11:54:44.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:54:44.701+0000 7feea8f29700 1 -- 172.21.15.93:0/550517567 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feea40684e0 con 0x7feea40fea80 2023-07-14T11:54:44.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:54:44.702+0000 7feea8f29700 1 -- 172.21.15.93:0/550517567 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feea410a250 con 0x7feea40fea80 2023-07-14T11:54:44.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:54:44.703+0000 7fee937fe700 1 -- 172.21.15.93:0/550517567 <== mon.2 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7fee98003f50 con 0x7feea40fea80 2023-07-14T11:54:44.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:54:44.703+0000 7fee937fe700 1 --2- 172.21.15.93:0/550517567 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fee8c0a10c0 0x7fee8c0a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:54:44.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:54:44.704+0000 7fee937fe700 1 -- 172.21.15.93:0/550517567 <== mon.2 v2:172.21.15.93:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7fee980cc4a0 con 0x7feea40fea80 2023-07-14T11:54:44.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:54:44.704+0000 7feea1d9b700 1 --2- 172.21.15.93:0/550517567 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fee8c0a10c0 0x7fee8c0a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:54:44.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:54:44.704+0000 7feea1d9b700 1 --2- 172.21.15.93:0/550517567 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fee8c0a10c0 0x7fee8c0a3570 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fee94005fd0 tx=0x7fee94005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:54:44.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:54:44.707+0000 7fee937fe700 1 -- 172.21.15.93:0/550517567 <== mon.2 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fee980976f0 con 0x7feea40fea80 2023-07-14T11:54:44.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:54:44.898+0000 7feea8f29700 1 -- 172.21.15.93:0/550517567 --> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7feea4060950 con 0x7fee8c0a10c0 2023-07-14T11:54:45.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:44 smithi093 ceph-mon[128922]: Detected new or changed devices on smithi017 2023-07-14T11:54:45.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:44 smithi093 ceph-mon[128922]: Adjusting osd_memory_target on smithi017 to 17272M 2023-07-14T11:54:45.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:44 smithi093 ceph-mon[128922]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:54:45.364 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:44 smithi093 ceph-mon[128922]: osdmap e9: 1 total, 1 up, 1 in 2023-07-14T11:54:45.364 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:44 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:54:45.364 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:44 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:54:45.364 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:44 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:45.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:44 smithi097 ceph-mon[129313]: Detected new or changed devices on smithi017 2023-07-14T11:54:45.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:44 smithi097 ceph-mon[129313]: Adjusting osd_memory_target on smithi017 to 17272M 2023-07-14T11:54:45.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:44 smithi097 ceph-mon[129313]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:54:45.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:44 smithi097 ceph-mon[129313]: osdmap e9: 1 total, 1 up, 1 in 2023-07-14T11:54:45.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:44 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:54:45.388 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:44 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:54:45.388 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:44 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:45.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:44 smithi017 ceph-mon[126339]: Detected new or changed devices on smithi017 2023-07-14T11:54:45.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:44 smithi017 ceph-mon[126339]: Adjusting osd_memory_target on smithi017 to 17272M 2023-07-14T11:54:45.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:44 smithi017 ceph-mon[126339]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:54:45.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:44 smithi017 ceph-mon[126339]: osdmap e9: 1 total, 1 up, 1 in 2023-07-14T11:54:45.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:44 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:54:45.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:44 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:54:45.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:44 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:46.020 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:45 smithi093 ceph-mon[128922]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:54:46.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:45 smithi097 ceph-mon[129313]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:54:46.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:45 smithi017 ceph-mon[126339]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:54:47.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:46 smithi093 ceph-mon[128922]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:54:47.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:46 smithi097 ceph-mon[129313]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:54:47.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:46 smithi017 ceph-mon[126339]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:54:48.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:47 smithi093 ceph-mon[128922]: from='client.? 172.21.15.93:0/3605334631' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f56b6337-d8a9-4411-b9d8-b6863dc98556"}]: dispatch 2023-07-14T11:54:48.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:47 smithi093 ceph-mon[128922]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f56b6337-d8a9-4411-b9d8-b6863dc98556"}]: dispatch 2023-07-14T11:54:48.364 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:47 smithi093 ceph-mon[128922]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f56b6337-d8a9-4411-b9d8-b6863dc98556"}]': finished 2023-07-14T11:54:48.364 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:47 smithi093 ceph-mon[128922]: osdmap e10: 2 total, 1 up, 2 in 2023-07-14T11:54:48.364 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:47 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T11:54:48.364 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:47 smithi093 ceph-mon[128922]: from='client.? 172.21.15.93:0/224554285' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:54:48.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:47 smithi097 ceph-mon[129313]: from='client.? 172.21.15.93:0/3605334631' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f56b6337-d8a9-4411-b9d8-b6863dc98556"}]: dispatch 2023-07-14T11:54:48.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:47 smithi097 ceph-mon[129313]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f56b6337-d8a9-4411-b9d8-b6863dc98556"}]: dispatch 2023-07-14T11:54:48.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:47 smithi097 ceph-mon[129313]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f56b6337-d8a9-4411-b9d8-b6863dc98556"}]': finished 2023-07-14T11:54:48.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:47 smithi097 ceph-mon[129313]: osdmap e10: 2 total, 1 up, 2 in 2023-07-14T11:54:48.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:47 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T11:54:48.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:47 smithi097 ceph-mon[129313]: from='client.? 172.21.15.93:0/224554285' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:54:48.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:47 smithi017 ceph-mon[126339]: from='client.? 172.21.15.93:0/3605334631' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f56b6337-d8a9-4411-b9d8-b6863dc98556"}]: dispatch 2023-07-14T11:54:48.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:47 smithi017 ceph-mon[126339]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f56b6337-d8a9-4411-b9d8-b6863dc98556"}]: dispatch 2023-07-14T11:54:48.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:47 smithi017 ceph-mon[126339]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f56b6337-d8a9-4411-b9d8-b6863dc98556"}]': finished 2023-07-14T11:54:48.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:47 smithi017 ceph-mon[126339]: osdmap e10: 2 total, 1 up, 2 in 2023-07-14T11:54:48.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:47 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T11:54:48.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:47 smithi017 ceph-mon[126339]: from='client.? 172.21.15.93:0/224554285' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:54:49.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:48 smithi093 ceph-mon[128922]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:54:49.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:48 smithi097 ceph-mon[129313]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:54:49.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:48 smithi017 ceph-mon[126339]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:54:51.244 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:50 smithi093 ceph-mon[128922]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:54:51.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:50 smithi097 ceph-mon[129313]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:54:51.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:50 smithi017 ceph-mon[126339]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:54:53.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:52 smithi093 ceph-mon[128922]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:54:53.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:52 smithi097 ceph-mon[129313]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:54:53.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:52 smithi017 ceph-mon[126339]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:54:54.215 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:53 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-07-14T11:54:54.215 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:53 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:54.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:53 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-07-14T11:54:54.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:53 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:54.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:53 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-07-14T11:54:54.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:53 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:54:55.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:54 smithi093 ceph-mon[128922]: Deploying daemon osd.1 on smithi093 2023-07-14T11:54:55.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:54 smithi093 ceph-mon[128922]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:54:55.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:54 smithi097 ceph-mon[129313]: Deploying daemon osd.1 on smithi093 2023-07-14T11:54:55.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:54 smithi097 ceph-mon[129313]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:54:55.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:54 smithi017 ceph-mon[126339]: Deploying daemon osd.1 on smithi093 2023-07-14T11:54:55.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:54 smithi017 ceph-mon[126339]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:54:57.344 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:56 smithi093 ceph-mon[128922]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:54:57.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:56 smithi097 ceph-mon[129313]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:54:57.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:56 smithi017 ceph-mon[126339]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:54:59.210 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:58 smithi093 ceph-mon[128922]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:54:59.211 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:58 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:59.211 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:58 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:59.211 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:54:58 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:54:59.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:58 smithi097 ceph-mon[129313]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:54:59.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:58 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:59.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:58 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:59.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:54:58 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:54:59.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:58 smithi017 ceph-mon[126339]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:54:59.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:58 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:59.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:58 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:54:59.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:54:58 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:55:01.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:01.064+0000 7fee937fe700 1 -- 172.21.15.93:0/550517567 <== mgr.14152 v2:172.21.15.17:6800/4237449858 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7feea4060950 con 0x7fee8c0a10c0 2023-07-14T11:55:01.047 INFO:teuthology.orchestra.run.smithi093.stdout:Created osd(s) 1 on host 'smithi093' 2023-07-14T11:55:01.049 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:01.067+0000 7feea8f29700 1 -- 172.21.15.93:0/550517567 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fee8c0a10c0 msgr2=0x7fee8c0a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:01.049 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:01.067+0000 7feea8f29700 1 --2- 172.21.15.93:0/550517567 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fee8c0a10c0 0x7fee8c0a3570 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fee94005fd0 tx=0x7fee94005d00 comp rx=0 tx=0).stop 2023-07-14T11:55:01.049 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:01.067+0000 7feea8f29700 1 -- 172.21.15.93:0/550517567 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feea40fea80 msgr2=0x7feea41947b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:01.049 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:01.067+0000 7feea8f29700 1 --2- 172.21.15.93:0/550517567 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feea40fea80 0x7feea41947b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fee98007b50 tx=0x7fee98007f10 comp rx=0 tx=0).stop 2023-07-14T11:55:01.050 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:01.067+0000 7feea8f29700 1 -- 172.21.15.93:0/550517567 shutdown_connections 2023-07-14T11:55:01.050 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:01.067+0000 7feea8f29700 1 --2- 172.21.15.93:0/550517567 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fee8c0a10c0 0x7fee8c0a3570 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:01.050 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:01.067+0000 7feea8f29700 1 --2- 172.21.15.93:0/550517567 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feea40fea80 0x7feea41947b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:01.050 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:01.067+0000 7feea8f29700 1 --2- 172.21.15.93:0/550517567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feea4100610 0x7feea419bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:01.050 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:01.067+0000 7feea8f29700 1 --2- 172.21.15.93:0/550517567 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feea40ffc80 0x7feea4194cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:01.050 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:01.067+0000 7feea8f29700 1 -- 172.21.15.93:0/550517567 >> 172.21.15.93:0/550517567 conn(0x7feea40fa0b0 msgr2=0x7feea40fc260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:01.051 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:01.067+0000 7feea8f29700 1 -- 172.21.15.93:0/550517567 shutdown_connections 2023-07-14T11:55:01.051 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:01.067+0000 7feea8f29700 1 -- 172.21.15.93:0/550517567 wait complete. 2023-07-14T11:55:01.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:00 smithi093 ceph-mon[128922]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:55:01.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:00 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:01.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:00 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:01.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:00 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:55:01.364 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:00 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:55:01.364 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:00 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:01.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:00 smithi097 ceph-mon[129313]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:55:01.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:00 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:01.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:00 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:01.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:00 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:55:01.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:00 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:55:01.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:00 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:01.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:00 smithi017 ceph-mon[126339]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:55:01.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:00 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:01.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:00 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:01.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:00 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:55:01.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:00 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:55:01.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:00 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:01.412 DEBUG:teuthology.orchestra.run.smithi093:osd.1> sudo journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@osd.1.service 2023-07-14T11:55:01.414 INFO:tasks.cephadm:Deploying osd.2 on smithi097 with /dev/vg_nvme/lv_4... 2023-07-14T11:55:01.415 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-07-14T11:55:01.448 INFO:journalctl@ceph.osd.1.smithi093.stdout:-- Logs begin at Fri 2023-07-14 11:24:16 UTC. -- 2023-07-14T11:55:01.977 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T11:55:02.242 INFO:journalctl@ceph.osd.1.smithi093.stdout:Jul 14 11:55:02 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-1[134027]: 2023-07-14T11:55:02.159+0000 7f117fcf6640 -1 osd.1 0 log_to_monitors true 2023-07-14T11:55:02.243 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:02 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:02.243 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:02 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:02.243 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:02 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:55:02.243 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:02 smithi093 ceph-mon[128922]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:55:02.351 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:02 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:02.351 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:02 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:02.352 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:02 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:55:02.352 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:02 smithi097 ceph-mon[129313]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:55:02.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:02 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:02.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:02 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:02.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:02 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:55:02.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:02 smithi017 ceph-mon[126339]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:55:03.269 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:03 smithi093 ceph-mon[128922]: from='osd.1 [v2:172.21.15.93:6800/3147203064,v1:172.21.15.93:6801/3147203064]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-14T11:55:03.270 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:03 smithi093 ceph-mon[128922]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-14T11:55:03.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:03 smithi097 ceph-mon[129313]: from='osd.1 [v2:172.21.15.93:6800/3147203064,v1:172.21.15.93:6801/3147203064]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-14T11:55:03.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:03 smithi097 ceph-mon[129313]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-14T11:55:03.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:03 smithi017 ceph-mon[126339]: from='osd.1 [v2:172.21.15.93:6800/3147203064,v1:172.21.15.93:6801/3147203064]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-14T11:55:03.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:03 smithi017 ceph-mon[126339]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-14T11:55:03.475 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-07-14T11:55:03.476 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-07-14T11:55:03.476 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2023-07-14T11:55:03.476 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2023-07-14T11:55:03.477 INFO:teuthology.orchestra.run.smithi097.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.01484 s, 707 MB/s 2023-07-14T11:55:03.477 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2023-07-14T11:55:03.929 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi097:vg_nvme/lv_4 2023-07-14T11:55:04.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:04 smithi097 ceph-mon[129313]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-07-14T11:55:04.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:04 smithi097 ceph-mon[129313]: osdmap e11: 2 total, 1 up, 2 in 2023-07-14T11:55:04.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:04 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T11:55:04.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:04 smithi097 ceph-mon[129313]: from='osd.1 [v2:172.21.15.93:6800/3147203064,v1:172.21.15.93:6801/3147203064]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-07-14T11:55:04.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:04 smithi097 ceph-mon[129313]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-07-14T11:55:04.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:04 smithi097 ceph-mon[129313]: Detected new or changed devices on smithi093 2023-07-14T11:55:04.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:04 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:04.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:04 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:04.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:04 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:55:04.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:04 smithi097 ceph-mon[129313]: Adjusting osd_memory_target on smithi093 to 17168M 2023-07-14T11:55:04.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:04 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:04.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:04 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:55:04.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:04 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:55:04.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:04 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:04.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:04 smithi097 ceph-mon[129313]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:55:04.363 INFO:journalctl@ceph.osd.1.smithi093.stdout:Jul 14 11:55:04 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-1[134027]: 2023-07-14T11:55:04.046+0000 7f117536e700 -1 osd.1 0 waiting for initial osdmap 2023-07-14T11:55:04.363 INFO:journalctl@ceph.osd.1.smithi093.stdout:Jul 14 11:55:04 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-1[134027]: 2023-07-14T11:55:04.048+0000 7f116e95e700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T11:55:04.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:04 smithi093 ceph-mon[128922]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-07-14T11:55:04.364 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:04 smithi093 ceph-mon[128922]: osdmap e11: 2 total, 1 up, 2 in 2023-07-14T11:55:04.364 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:04 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T11:55:04.364 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:04 smithi093 ceph-mon[128922]: from='osd.1 [v2:172.21.15.93:6800/3147203064,v1:172.21.15.93:6801/3147203064]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-07-14T11:55:04.364 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:04 smithi093 ceph-mon[128922]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-07-14T11:55:04.364 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:04 smithi093 ceph-mon[128922]: Detected new or changed devices on smithi093 2023-07-14T11:55:04.364 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:04 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:04.364 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:04 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:04.365 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:04 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:55:04.365 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:04 smithi093 ceph-mon[128922]: Adjusting osd_memory_target on smithi093 to 17168M 2023-07-14T11:55:04.365 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:04 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:04.365 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:04 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:55:04.365 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:04 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:55:04.365 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:04 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:04.365 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:04 smithi093 ceph-mon[128922]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:55:04.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:04 smithi017 ceph-mon[126339]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-07-14T11:55:04.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:04 smithi017 ceph-mon[126339]: osdmap e11: 2 total, 1 up, 2 in 2023-07-14T11:55:04.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:04 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T11:55:04.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:04 smithi017 ceph-mon[126339]: from='osd.1 [v2:172.21.15.93:6800/3147203064,v1:172.21.15.93:6801/3147203064]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-07-14T11:55:04.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:04 smithi017 ceph-mon[126339]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-07-14T11:55:04.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:04 smithi017 ceph-mon[126339]: Detected new or changed devices on smithi093 2023-07-14T11:55:04.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:04 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:04.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:04 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:04.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:04 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:55:04.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:04 smithi017 ceph-mon[126339]: Adjusting osd_memory_target on smithi093 to 17168M 2023-07-14T11:55:04.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:04 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:04.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:04 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:55:04.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:04 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:55:04.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:04 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:04.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:04 smithi017 ceph-mon[126339]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:55:04.480 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T11:55:05.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:05 smithi093 ceph-mon[128922]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-07-14T11:55:05.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:05 smithi093 ceph-mon[128922]: osdmap e12: 2 total, 1 up, 2 in 2023-07-14T11:55:05.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:05 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T11:55:05.364 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:05 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T11:55:05.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:05 smithi097 ceph-mon[129313]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-07-14T11:55:05.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:05 smithi097 ceph-mon[129313]: osdmap e12: 2 total, 1 up, 2 in 2023-07-14T11:55:05.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:05 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T11:55:05.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:05 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T11:55:05.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:05 smithi017 ceph-mon[126339]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-07-14T11:55:05.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:05 smithi017 ceph-mon[126339]: osdmap e12: 2 total, 1 up, 2 in 2023-07-14T11:55:05.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:05 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T11:55:05.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:05 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T11:55:05.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:05.739+0000 7f6e465e4700 1 -- 172.21.15.97:0/3434035161 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e400fed00 msgr2=0x7f6e400ff170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:05.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:05.739+0000 7f6e465e4700 1 --2- 172.21.15.97:0/3434035161 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e400fed00 0x7f6e400ff170 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6e30009990 tx=0x7f6e30009ca0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:05.739+0000 7f6e465e4700 1 -- 172.21.15.97:0/3434035161 shutdown_connections 2023-07-14T11:55:05.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:05.739+0000 7f6e465e4700 1 --2- 172.21.15.97:0/3434035161 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e401000a0 0x7f6e401004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:05.739+0000 7f6e465e4700 1 --2- 172.21.15.97:0/3434035161 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e400ff6b0 0x7f6e40106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:05.739+0000 7f6e465e4700 1 --2- 172.21.15.97:0/3434035161 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e400fed00 0x7f6e400ff170 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:05.739+0000 7f6e465e4700 1 -- 172.21.15.97:0/3434035161 >> 172.21.15.97:0/3434035161 conn(0x7f6e400f9860 msgr2=0x7f6e400fbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:05.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:05.739+0000 7f6e465e4700 1 -- 172.21.15.97:0/3434035161 shutdown_connections 2023-07-14T11:55:05.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:05.739+0000 7f6e465e4700 1 -- 172.21.15.97:0/3434035161 wait complete. 2023-07-14T11:55:05.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:05.739+0000 7f6e465e4700 1 Processor -- start 2023-07-14T11:55:05.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:05.739+0000 7f6e465e4700 1 -- start start 2023-07-14T11:55:05.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:05.740+0000 7f6e465e4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e400fed00 0x7f6e401959c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:05.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:05.740+0000 7f6e465e4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e400ff6b0 0x7f6e40195f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:05.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:05.740+0000 7f6e465e4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e401000a0 0x7f6e4019cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:05.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:05.740+0000 7f6e465e4700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4019d550 con 0x7f6e400fed00 2023-07-14T11:55:05.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:05.740+0000 7f6e465e4700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4019d690 con 0x7f6e401000a0 2023-07-14T11:55:05.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:05.740+0000 7f6e465e4700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4019d820 con 0x7f6e400ff6b0 2023-07-14T11:55:05.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:05.740+0000 7f6e3f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e400ff6b0 0x7f6e40195f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:05.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:05.740+0000 7f6e3f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e400ff6b0 0x7f6e40195f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42212/0 (socket says 172.21.15.97:42212) 2023-07-14T11:55:05.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:05.740+0000 7f6e3f7fe700 1 -- 172.21.15.97:0/3603484641 learned_addr learned my addr 172.21.15.97:0/3603484641 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-14T11:55:05.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:05.740+0000 7f6e3ffff700 1 --2- 172.21.15.97:0/3603484641 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e400fed00 0x7f6e401959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:05.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:05.740+0000 7f6e3f7fe700 1 -- 172.21.15.97:0/3603484641 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e401000a0 msgr2=0x7f6e4019cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:05.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:05.740+0000 7f6e44b81700 1 --2- 172.21.15.97:0/3603484641 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e401000a0 0x7f6e4019cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:05.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:05.740+0000 7f6e3f7fe700 1 --2- 172.21.15.97:0/3603484641 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e401000a0 0x7f6e4019cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:05.740+0000 7f6e3f7fe700 1 -- 172.21.15.97:0/3603484641 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e400fed00 msgr2=0x7f6e401959c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:05.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:05.740+0000 7f6e3f7fe700 1 --2- 172.21.15.97:0/3603484641 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e400fed00 0x7f6e401959c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:05.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:05.740+0000 7f6e3f7fe700 1 -- 172.21.15.97:0/3603484641 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e300095f0 con 0x7f6e400ff6b0 2023-07-14T11:55:05.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:05.740+0000 7f6e3ffff700 1 --2- 172.21.15.97:0/3603484641 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e400fed00 0x7f6e401959c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:55:05.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:05.740+0000 7f6e44b81700 1 --2- 172.21.15.97:0/3603484641 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e401000a0 0x7f6e4019cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:55:05.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:05.740+0000 7f6e3f7fe700 1 --2- 172.21.15.97:0/3603484641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e400ff6b0 0x7f6e40195f00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6e30005910 tx=0x7f6e3000f7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:05.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:05.740+0000 7f6e3d7fa700 1 -- 172.21.15.97:0/3603484641 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3001d070 con 0x7f6e400ff6b0 2023-07-14T11:55:05.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:05.740+0000 7f6e3d7fa700 1 -- 172.21.15.97:0/3603484641 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e3000fc60 con 0x7f6e400ff6b0 2023-07-14T11:55:05.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:05.740+0000 7f6e465e4700 1 -- 172.21.15.97:0/3603484641 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e4019daa0 con 0x7f6e400ff6b0 2023-07-14T11:55:05.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:05.740+0000 7f6e3d7fa700 1 -- 172.21.15.97:0/3603484641 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e300176e0 con 0x7f6e400ff6b0 2023-07-14T11:55:05.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:05.741+0000 7f6e465e4700 1 -- 172.21.15.97:0/3603484641 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e4019df30 con 0x7f6e400ff6b0 2023-07-14T11:55:05.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:05.741+0000 7f6e465e4700 1 -- 172.21.15.97:0/3603484641 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e4010a3c0 con 0x7f6e400ff6b0 2023-07-14T11:55:05.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:05.743+0000 7f6e3d7fa700 1 -- 172.21.15.97:0/3603484641 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7f6e30022470 con 0x7f6e400ff6b0 2023-07-14T11:55:05.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:05.743+0000 7f6e3d7fa700 1 --2- 172.21.15.97:0/3603484641 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f6e2c0a10c0 0x7f6e2c0a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:05.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:05.743+0000 7f6e3d7fa700 1 -- 172.21.15.97:0/3603484641 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f6e300cc760 con 0x7f6e400ff6b0 2023-07-14T11:55:05.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:05.744+0000 7f6e3ffff700 1 --2- 172.21.15.97:0/3603484641 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f6e2c0a10c0 0x7f6e2c0a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:05.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:05.744+0000 7f6e3ffff700 1 --2- 172.21.15.97:0/3603484641 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f6e2c0a10c0 0x7f6e2c0a3570 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6e40067db0 tx=0x7f6e28008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:05.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:05.747+0000 7f6e3d7fa700 1 -- 172.21.15.97:0/3603484641 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6e30097790 con 0x7f6e400ff6b0 2023-07-14T11:55:05.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:05.938+0000 7f6e465e4700 1 -- 172.21.15.97:0/3603484641 --> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e400fe320 con 0x7f6e2c0a10c0 2023-07-14T11:55:06.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:06 smithi093 ceph-mon[128922]: purged_snaps scrub starts 2023-07-14T11:55:06.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:06 smithi093 ceph-mon[128922]: purged_snaps scrub ok 2023-07-14T11:55:06.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:06 smithi093 ceph-mon[128922]: osd.1 [v2:172.21.15.93:6800/3147203064,v1:172.21.15.93:6801/3147203064] boot 2023-07-14T11:55:06.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:06 smithi093 ceph-mon[128922]: osdmap e13: 2 total, 2 up, 2 in 2023-07-14T11:55:06.364 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:06 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T11:55:06.364 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:06 smithi093 ceph-mon[128922]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:55:06.364 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:06 smithi093 ceph-mon[128922]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:55:06.364 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:06 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:55:06.364 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:06 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:55:06.365 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:06 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:55:06.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:06 smithi097 ceph-mon[129313]: purged_snaps scrub starts 2023-07-14T11:55:06.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:06 smithi097 ceph-mon[129313]: purged_snaps scrub ok 2023-07-14T11:55:06.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:06 smithi097 ceph-mon[129313]: osd.1 [v2:172.21.15.93:6800/3147203064,v1:172.21.15.93:6801/3147203064] boot 2023-07-14T11:55:06.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:06 smithi097 ceph-mon[129313]: osdmap e13: 2 total, 2 up, 2 in 2023-07-14T11:55:06.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:06 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T11:55:06.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:06 smithi097 ceph-mon[129313]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:55:06.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:06 smithi097 ceph-mon[129313]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:55:06.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:06 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:55:06.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:06 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:55:06.388 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:06 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:55:06.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:06 smithi017 ceph-mon[126339]: purged_snaps scrub starts 2023-07-14T11:55:06.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:06 smithi017 ceph-mon[126339]: purged_snaps scrub ok 2023-07-14T11:55:06.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:06 smithi017 ceph-mon[126339]: osd.1 [v2:172.21.15.93:6800/3147203064,v1:172.21.15.93:6801/3147203064] boot 2023-07-14T11:55:06.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:06 smithi017 ceph-mon[126339]: osdmap e13: 2 total, 2 up, 2 in 2023-07-14T11:55:06.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:06 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T11:55:06.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:06 smithi017 ceph-mon[126339]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T11:55:06.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:06 smithi017 ceph-mon[126339]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:55:06.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:06 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:55:06.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:06 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:55:06.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:06 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:55:07.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:07 smithi093 ceph-mon[128922]: osdmap e14: 2 total, 2 up, 2 in 2023-07-14T11:55:07.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:07 smithi097 ceph-mon[129313]: osdmap e14: 2 total, 2 up, 2 in 2023-07-14T11:55:07.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:07 smithi017 ceph-mon[126339]: osdmap e14: 2 total, 2 up, 2 in 2023-07-14T11:55:09.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:08 smithi017 ceph-mon[126339]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:55:09.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:08 smithi017 ceph-mon[126339]: from='client.? 172.21.15.97:0/914167528' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc34bb7f-551f-46e1-99c0-59924d6c1346"}]: dispatch 2023-07-14T11:55:09.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:08 smithi017 ceph-mon[126339]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc34bb7f-551f-46e1-99c0-59924d6c1346"}]: dispatch 2023-07-14T11:55:09.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:08 smithi017 ceph-mon[126339]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc34bb7f-551f-46e1-99c0-59924d6c1346"}]': finished 2023-07-14T11:55:09.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:08 smithi017 ceph-mon[126339]: osdmap e15: 3 total, 2 up, 3 in 2023-07-14T11:55:09.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:08 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T11:55:09.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:08 smithi017 ceph-mon[126339]: from='client.? 172.21.15.97:0/3811498261' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:55:09.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:08 smithi093 ceph-mon[128922]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:55:09.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:08 smithi093 ceph-mon[128922]: from='client.? 172.21.15.97:0/914167528' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc34bb7f-551f-46e1-99c0-59924d6c1346"}]: dispatch 2023-07-14T11:55:09.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:08 smithi093 ceph-mon[128922]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc34bb7f-551f-46e1-99c0-59924d6c1346"}]: dispatch 2023-07-14T11:55:09.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:08 smithi093 ceph-mon[128922]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc34bb7f-551f-46e1-99c0-59924d6c1346"}]': finished 2023-07-14T11:55:09.364 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:08 smithi093 ceph-mon[128922]: osdmap e15: 3 total, 2 up, 3 in 2023-07-14T11:55:09.364 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:08 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T11:55:09.364 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:08 smithi093 ceph-mon[128922]: from='client.? 172.21.15.97:0/3811498261' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:55:09.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:08 smithi097 ceph-mon[129313]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:55:09.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:08 smithi097 ceph-mon[129313]: from='client.? 172.21.15.97:0/914167528' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc34bb7f-551f-46e1-99c0-59924d6c1346"}]: dispatch 2023-07-14T11:55:09.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:08 smithi097 ceph-mon[129313]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc34bb7f-551f-46e1-99c0-59924d6c1346"}]: dispatch 2023-07-14T11:55:09.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:08 smithi097 ceph-mon[129313]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc34bb7f-551f-46e1-99c0-59924d6c1346"}]': finished 2023-07-14T11:55:09.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:08 smithi097 ceph-mon[129313]: osdmap e15: 3 total, 2 up, 3 in 2023-07-14T11:55:09.388 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:08 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T11:55:09.388 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:08 smithi097 ceph-mon[129313]: from='client.? 172.21.15.97:0/3811498261' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:55:11.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:10 smithi017 ceph-mon[126339]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:55:11.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:10 smithi093 ceph-mon[128922]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:55:11.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:10 smithi097 ceph-mon[129313]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:55:13.231 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:12 smithi097 ceph-mon[129313]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:55:13.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:12 smithi093 ceph-mon[128922]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:55:13.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:12 smithi017 ceph-mon[126339]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:55:15.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:14 smithi097 ceph-mon[129313]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:55:15.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:14 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-07-14T11:55:15.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:14 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:55:15.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:14 smithi093 ceph-mon[128922]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:55:15.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:14 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-07-14T11:55:15.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:14 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:55:15.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:14 smithi017 ceph-mon[126339]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:55:15.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:14 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-07-14T11:55:15.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:14 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:55:16.168 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:15 smithi097 ceph-mon[129313]: Deploying daemon osd.2 on smithi097 2023-07-14T11:55:16.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:15 smithi093 ceph-mon[128922]: Deploying daemon osd.2 on smithi097 2023-07-14T11:55:16.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:15 smithi017 ceph-mon[126339]: Deploying daemon osd.2 on smithi097 2023-07-14T11:55:17.157 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:16 smithi097 ceph-mon[129313]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:55:17.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:16 smithi093 ceph-mon[128922]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:55:17.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:16 smithi017 ceph-mon[126339]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:55:19.111 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:18 smithi097 ceph-mon[129313]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:55:19.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:18 smithi093 ceph-mon[128922]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:55:19.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:18 smithi017 ceph-mon[126339]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:55:20.278 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:20 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:20.279 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:20 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:20.279 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:20 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:55:20.279 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:20 smithi097 ceph-mon[129313]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:55:20.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:20 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:20.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:20 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:20.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:20 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:55:20.614 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:20 smithi093 ceph-mon[128922]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:55:20.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:20 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:20.647 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:20 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:20.648 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:20 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:55:20.648 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:20 smithi017 ceph-mon[126339]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:55:21.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:21.854+0000 7f6e3d7fa700 1 -- 172.21.15.97:0/3603484641 <== mgr.14152 v2:172.21.15.17:6800/4237449858 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6e400fe320 con 0x7f6e2c0a10c0 2023-07-14T11:55:21.838 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 2 on host 'smithi097' 2023-07-14T11:55:21.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:21.857+0000 7f6e465e4700 1 -- 172.21.15.97:0/3603484641 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f6e2c0a10c0 msgr2=0x7f6e2c0a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:21.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:21.857+0000 7f6e465e4700 1 --2- 172.21.15.97:0/3603484641 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f6e2c0a10c0 0x7f6e2c0a3570 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6e40067db0 tx=0x7f6e28008040 comp rx=0 tx=0).stop 2023-07-14T11:55:21.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:21.857+0000 7f6e465e4700 1 -- 172.21.15.97:0/3603484641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e400ff6b0 msgr2=0x7f6e40195f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:21.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:21.857+0000 7f6e465e4700 1 --2- 172.21.15.97:0/3603484641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e400ff6b0 0x7f6e40195f00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6e30005910 tx=0x7f6e3000f7b0 comp rx=0 tx=0).stop 2023-07-14T11:55:21.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:21.857+0000 7f6e465e4700 1 -- 172.21.15.97:0/3603484641 shutdown_connections 2023-07-14T11:55:21.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:21.857+0000 7f6e465e4700 1 --2- 172.21.15.97:0/3603484641 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f6e2c0a10c0 0x7f6e2c0a3570 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:21.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:21.857+0000 7f6e465e4700 1 --2- 172.21.15.97:0/3603484641 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e401000a0 0x7f6e4019cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:21.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:21.857+0000 7f6e465e4700 1 --2- 172.21.15.97:0/3603484641 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e400fed00 0x7f6e401959c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:21.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:21.857+0000 7f6e465e4700 1 --2- 172.21.15.97:0/3603484641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e400ff6b0 0x7f6e40195f00 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:21.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:21.857+0000 7f6e465e4700 1 -- 172.21.15.97:0/3603484641 >> 172.21.15.97:0/3603484641 conn(0x7f6e400f9860 msgr2=0x7f6e400fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:21.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:21.857+0000 7f6e465e4700 1 -- 172.21.15.97:0/3603484641 shutdown_connections 2023-07-14T11:55:21.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:55:21.857+0000 7f6e465e4700 1 -- 172.21.15.97:0/3603484641 wait complete. 2023-07-14T11:55:22.255 DEBUG:teuthology.orchestra.run.smithi097:osd.2> sudo journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@osd.2.service 2023-07-14T11:55:22.258 INFO:tasks.cephadm:Deploying osd.3 on smithi017 with /dev/vg_nvme/lv_3... 2023-07-14T11:55:22.258 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-07-14T11:55:22.288 INFO:journalctl@ceph.osd.2.smithi097.stdout:-- Logs begin at Fri 2023-07-14 11:24:16 UTC. -- 2023-07-14T11:55:22.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:22 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:22.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:22 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:22.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:22 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:55:22.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:22 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:55:22.614 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:22 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:22.614 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:22 smithi093 ceph-mon[128922]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:55:22.614 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:22 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:22.614 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:22 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:22.614 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:22 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:55:22.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:22 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:22.637 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:22 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:22.637 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:22 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:55:22.637 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:22 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:55:22.637 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:22 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:22.637 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:22 smithi097 ceph-mon[129313]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:55:22.637 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:22 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:22.638 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:22 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:22.638 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:22 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:55:22.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:22 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:22.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:22 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:22.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:22 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:55:22.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:22 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:55:22.648 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:22 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:22.648 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:22 smithi017 ceph-mon[126339]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:55:22.649 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:22 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:22.649 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:22 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:22.649 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:22 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:55:22.999 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T11:55:23.386 INFO:journalctl@ceph.osd.2.smithi097.stdout:Jul 14 11:55:23 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-2[135091]: 2023-07-14T11:55:23.125+0000 7f9554d7f640 -1 osd.2 0 log_to_monitors true 2023-07-14T11:55:24.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:23 smithi017 ceph-mon[126339]: from='osd.2 [v2:172.21.15.97:6800/1877269815,v1:172.21.15.97:6801/1877269815]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-14T11:55:24.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:23 smithi017 ceph-mon[126339]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-14T11:55:24.269 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:23 smithi097 ceph-mon[129313]: from='osd.2 [v2:172.21.15.97:6800/1877269815,v1:172.21.15.97:6801/1877269815]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-14T11:55:24.270 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:23 smithi097 ceph-mon[129313]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-14T11:55:24.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:23 smithi093 ceph-mon[128922]: from='osd.2 [v2:172.21.15.97:6800/1877269815,v1:172.21.15.97:6801/1877269815]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-14T11:55:24.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:23 smithi093 ceph-mon[128922]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-14T11:55:25.065 INFO:teuthology.orchestra.run.smithi017.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-07-14T11:55:25.066 INFO:teuthology.orchestra.run.smithi017.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-07-14T11:55:25.066 INFO:teuthology.orchestra.run.smithi017.stderr: stderr: 10+0 records in 2023-07-14T11:55:25.066 INFO:teuthology.orchestra.run.smithi017.stderr:10+0 records out 2023-07-14T11:55:25.066 INFO:teuthology.orchestra.run.smithi017.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0301418 s, 348 MB/s 2023-07-14T11:55:25.066 INFO:teuthology.orchestra.run.smithi017.stderr:--> Zapping successful for: 2023-07-14T11:55:25.279 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:24 smithi017 ceph-mon[126339]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:55:25.279 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:24 smithi017 ceph-mon[126339]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-07-14T11:55:25.279 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:24 smithi017 ceph-mon[126339]: osdmap e16: 3 total, 2 up, 3 in 2023-07-14T11:55:25.280 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:24 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T11:55:25.280 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:24 smithi017 ceph-mon[126339]: from='osd.2 [v2:172.21.15.97:6800/1877269815,v1:172.21.15.97:6801/1877269815]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-07-14T11:55:25.280 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:24 smithi017 ceph-mon[126339]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-07-14T11:55:25.280 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:24 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:25.280 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:24 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:25.280 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:24 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:55:25.280 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:24 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:25.281 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:24 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:55:25.281 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:24 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:55:25.281 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:24 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:25.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:24 smithi093 ceph-mon[128922]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:55:25.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:24 smithi093 ceph-mon[128922]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-07-14T11:55:25.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:24 smithi093 ceph-mon[128922]: osdmap e16: 3 total, 2 up, 3 in 2023-07-14T11:55:25.364 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:24 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T11:55:25.364 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:24 smithi093 ceph-mon[128922]: from='osd.2 [v2:172.21.15.97:6800/1877269815,v1:172.21.15.97:6801/1877269815]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-07-14T11:55:25.364 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:24 smithi093 ceph-mon[128922]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-07-14T11:55:25.364 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:24 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:25.364 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:24 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:25.364 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:24 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:55:25.364 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:24 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:25.365 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:24 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:55:25.365 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:24 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:55:25.365 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:24 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:25.386 INFO:journalctl@ceph.osd.2.smithi097.stdout:Jul 14 11:55:24 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-2[135091]: 2023-07-14T11:55:24.930+0000 7f954a3f7700 -1 osd.2 0 waiting for initial osdmap 2023-07-14T11:55:25.386 INFO:journalctl@ceph.osd.2.smithi097.stdout:Jul 14 11:55:24 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-2[135091]: 2023-07-14T11:55:24.934+0000 7f95441e8700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T11:55:25.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:24 smithi097 ceph-mon[129313]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:55:25.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:24 smithi097 ceph-mon[129313]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-07-14T11:55:25.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:24 smithi097 ceph-mon[129313]: osdmap e16: 3 total, 2 up, 3 in 2023-07-14T11:55:25.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:24 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T11:55:25.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:24 smithi097 ceph-mon[129313]: from='osd.2 [v2:172.21.15.97:6800/1877269815,v1:172.21.15.97:6801/1877269815]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-07-14T11:55:25.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:24 smithi097 ceph-mon[129313]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-07-14T11:55:25.388 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:24 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:25.388 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:24 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:25.388 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:24 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:55:25.388 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:24 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:25.388 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:24 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:55:25.388 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:24 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:55:25.388 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:24 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:25.574 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi017:vg_nvme/lv_3 2023-07-14T11:55:26.307 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T11:55:26.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:25 smithi093 ceph-mon[128922]: Detected new or changed devices on smithi097 2023-07-14T11:55:26.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:25 smithi093 ceph-mon[128922]: Adjusting osd_memory_target on smithi097 to 17168M 2023-07-14T11:55:26.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:25 smithi093 ceph-mon[128922]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2023-07-14T11:55:26.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:25 smithi093 ceph-mon[128922]: osdmap e17: 3 total, 2 up, 3 in 2023-07-14T11:55:26.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:25 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T11:55:26.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:25 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T11:55:26.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:25 smithi097 ceph-mon[129313]: Detected new or changed devices on smithi097 2023-07-14T11:55:26.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:25 smithi097 ceph-mon[129313]: Adjusting osd_memory_target on smithi097 to 17168M 2023-07-14T11:55:26.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:25 smithi097 ceph-mon[129313]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2023-07-14T11:55:26.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:25 smithi097 ceph-mon[129313]: osdmap e17: 3 total, 2 up, 3 in 2023-07-14T11:55:26.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:25 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T11:55:26.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:25 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T11:55:26.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:25 smithi017 ceph-mon[126339]: Detected new or changed devices on smithi097 2023-07-14T11:55:26.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:25 smithi017 ceph-mon[126339]: Adjusting osd_memory_target on smithi097 to 17168M 2023-07-14T11:55:26.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:25 smithi017 ceph-mon[126339]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2023-07-14T11:55:26.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:25 smithi017 ceph-mon[126339]: osdmap e17: 3 total, 2 up, 3 in 2023-07-14T11:55:26.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:25 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T11:55:26.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:25 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T11:55:27.285 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:26 smithi017 ceph-mon[126339]: purged_snaps scrub starts 2023-07-14T11:55:27.286 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:26 smithi017 ceph-mon[126339]: purged_snaps scrub ok 2023-07-14T11:55:27.286 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:26 smithi017 ceph-mon[126339]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:55:27.286 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:26 smithi017 ceph-mon[126339]: osd.2 [v2:172.21.15.97:6800/1877269815,v1:172.21.15.97:6801/1877269815] boot 2023-07-14T11:55:27.286 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:26 smithi017 ceph-mon[126339]: osdmap e18: 3 total, 3 up, 3 in 2023-07-14T11:55:27.286 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:26 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T11:55:27.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:26 smithi093 ceph-mon[128922]: purged_snaps scrub starts 2023-07-14T11:55:27.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:26 smithi093 ceph-mon[128922]: purged_snaps scrub ok 2023-07-14T11:55:27.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:26 smithi093 ceph-mon[128922]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:55:27.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:26 smithi093 ceph-mon[128922]: osd.2 [v2:172.21.15.97:6800/1877269815,v1:172.21.15.97:6801/1877269815] boot 2023-07-14T11:55:27.364 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:26 smithi093 ceph-mon[128922]: osdmap e18: 3 total, 3 up, 3 in 2023-07-14T11:55:27.364 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:26 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T11:55:27.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:26 smithi097 ceph-mon[129313]: purged_snaps scrub starts 2023-07-14T11:55:27.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:26 smithi097 ceph-mon[129313]: purged_snaps scrub ok 2023-07-14T11:55:27.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:26 smithi097 ceph-mon[129313]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T11:55:27.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:26 smithi097 ceph-mon[129313]: osd.2 [v2:172.21.15.97:6800/1877269815,v1:172.21.15.97:6801/1877269815] boot 2023-07-14T11:55:27.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:26 smithi097 ceph-mon[129313]: osdmap e18: 3 total, 3 up, 3 in 2023-07-14T11:55:27.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:26 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T11:55:27.935 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:27.953+0000 7f51afc35700 1 -- 172.21.15.17:0/853900512 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51a8074540 msgr2=0x7f51a80729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:27.936 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:27.953+0000 7f51afc35700 1 --2- 172.21.15.17:0/853900512 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51a8074540 0x7f51a80729a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f51a4009990 tx=0x7f51a4009ca0 comp rx=0 tx=0).stop 2023-07-14T11:55:27.936 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:27.953+0000 7f51afc35700 1 -- 172.21.15.17:0/853900512 shutdown_connections 2023-07-14T11:55:27.936 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:27.953+0000 7f51afc35700 1 --2- 172.21.15.17:0/853900512 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51a8074540 0x7f51a80729a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:27.937 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:27.953+0000 7f51afc35700 1 --2- 172.21.15.17:0/853900512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51a8107a60 0x7f51a8109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:27.937 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:27.953+0000 7f51afc35700 1 --2- 172.21.15.17:0/853900512 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51a8072f70 0x7f51a80733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:27.937 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:27.953+0000 7f51afc35700 1 -- 172.21.15.17:0/853900512 >> 172.21.15.17:0/853900512 conn(0x7f51a80fb1a0 msgr2=0x7f51a80fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:27.937 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:27.954+0000 7f51afc35700 1 -- 172.21.15.17:0/853900512 shutdown_connections 2023-07-14T11:55:27.937 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:27.954+0000 7f51afc35700 1 -- 172.21.15.17:0/853900512 wait complete. 2023-07-14T11:55:27.939 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:27.955+0000 7f51afc35700 1 Processor -- start 2023-07-14T11:55:27.939 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:27.955+0000 7f51afc35700 1 -- start start 2023-07-14T11:55:27.940 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:27.955+0000 7f51afc35700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51a8072f70 0x7f51a8197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:27.940 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:27.955+0000 7f51afc35700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51a8074540 0x7f51a8198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:27.940 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:27.955+0000 7f51afc35700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51a8107a60 0x7f51a819f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:27.940 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:27.955+0000 7f51afc35700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a819f650 con 0x7f51a8107a60 2023-07-14T11:55:27.941 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:27.955+0000 7f51afc35700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a819f790 con 0x7f51a8072f70 2023-07-14T11:55:27.941 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:27.955+0000 7f51afc35700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a819f920 con 0x7f51a8074540 2023-07-14T11:55:27.941 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:27.955+0000 7f51ad9d1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51a8072f70 0x7f51a8197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:27.941 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:27.955+0000 7f51ad9d1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51a8072f70 0x7f51a8197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.17:51232/0 (socket says 172.21.15.17:51232) 2023-07-14T11:55:27.942 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:27.955+0000 7f51ad9d1700 1 -- 172.21.15.17:0/3729445776 learned_addr learned my addr 172.21.15.17:0/3729445776 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:55:27.942 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:27.955+0000 7f51ae1d2700 1 --2- 172.21.15.17:0/3729445776 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51a8107a60 0x7f51a819f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:27.942 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:27.955+0000 7f51ad1d0700 1 --2- 172.21.15.17:0/3729445776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51a8074540 0x7f51a8198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:27.942 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:27.956+0000 7f51ad9d1700 1 -- 172.21.15.17:0/3729445776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51a8074540 msgr2=0x7f51a8198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:27.943 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:27.956+0000 7f51ad9d1700 1 --2- 172.21.15.17:0/3729445776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51a8074540 0x7f51a8198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:27.943 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:27.956+0000 7f51ad9d1700 1 -- 172.21.15.17:0/3729445776 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51a8107a60 msgr2=0x7f51a819f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:27.943 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:27.956+0000 7f51ad9d1700 1 --2- 172.21.15.17:0/3729445776 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51a8107a60 0x7f51a819f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:27.943 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:27.956+0000 7f51ad9d1700 1 -- 172.21.15.17:0/3729445776 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a40095f0 con 0x7f51a8072f70 2023-07-14T11:55:27.943 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:27.956+0000 7f51ad9d1700 1 --2- 172.21.15.17:0/3729445776 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51a8072f70 0x7f51a8197ac0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f51a4004730 tx=0x7f51a4004760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:27.944 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:27.956+0000 7f519effd700 1 -- 172.21.15.17:0/3729445776 <== mon.2 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a401d070 con 0x7f51a8072f70 2023-07-14T11:55:27.944 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:27.956+0000 7f51afc35700 1 -- 172.21.15.17:0/3729445776 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51a819fba0 con 0x7f51a8072f70 2023-07-14T11:55:27.944 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:27.957+0000 7f519effd700 1 -- 172.21.15.17:0/3729445776 <== mon.2 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51a4004c90 con 0x7f51a8072f70 2023-07-14T11:55:27.944 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:27.957+0000 7f51afc35700 1 -- 172.21.15.17:0/3729445776 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51a819ffb0 con 0x7f51a8072f70 2023-07-14T11:55:27.944 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:27.957+0000 7f519effd700 1 -- 172.21.15.17:0/3729445776 <== mon.2 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a4005490 con 0x7f51a8072f70 2023-07-14T11:55:27.945 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:27.958+0000 7f51afc35700 1 -- 172.21.15.17:0/3729445776 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51a8106110 con 0x7f51a8072f70 2023-07-14T11:55:27.945 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:27.960+0000 7f519effd700 1 -- 172.21.15.17:0/3729445776 <== mon.2 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7f51a400fbc0 con 0x7f51a8072f70 2023-07-14T11:55:27.945 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:27.961+0000 7f519effd700 1 --2- 172.21.15.17:0/3729445776 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f51940a10c0 0x7f51940a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:27.946 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:27.961+0000 7f519effd700 1 -- 172.21.15.17:0/3729445776 <== mon.2 v2:172.21.15.93:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 2851+0+0 (secure 0 0 0) 0x7f51a40cc550 con 0x7f51a8072f70 2023-07-14T11:55:27.946 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:27.961+0000 7f51ad1d0700 1 --2- 172.21.15.17:0/3729445776 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f51940a10c0 0x7f51940a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:27.946 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:27.962+0000 7f51ad1d0700 1 --2- 172.21.15.17:0/3729445776 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f51940a10c0 0x7f51940a3570 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5198005950 tx=0x7f519800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:27.946 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:27.963+0000 7f519effd700 1 -- 172.21.15.17:0/3729445776 <== mon.2 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f51a40972c0 con 0x7f51a8072f70 2023-07-14T11:55:28.163 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:28.181+0000 7f51afc35700 1 -- 172.21.15.17:0/3729445776 --> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f51a8060950 con 0x7f51940a10c0 2023-07-14T11:55:28.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:28 smithi093 ceph-mon[128922]: osdmap e19: 3 total, 3 up, 3 in 2023-07-14T11:55:28.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:28 smithi097 ceph-mon[129313]: osdmap e19: 3 total, 3 up, 3 in 2023-07-14T11:55:28.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:28 smithi017 ceph-mon[126339]: osdmap e19: 3 total, 3 up, 3 in 2023-07-14T11:55:29.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:29 smithi093 ceph-mon[128922]: pgmap v80: 0 pgs: ; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-14T11:55:29.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:29 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-07-14T11:55:29.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:29 smithi093 ceph-mon[128922]: from='client.24188 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:55:29.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:29 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:55:29.614 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:29 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:55:29.614 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:29 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:55:29.614 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:29 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-07-14T11:55:29.614 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:29 smithi093 ceph-mon[128922]: osdmap e20: 3 total, 3 up, 3 in 2023-07-14T11:55:29.614 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:29 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-07-14T11:55:29.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:29 smithi097 ceph-mon[129313]: pgmap v80: 0 pgs: ; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-14T11:55:29.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:29 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-07-14T11:55:29.637 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:29 smithi097 ceph-mon[129313]: from='client.24188 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:55:29.637 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:29 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:55:29.637 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:29 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:55:29.637 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:29 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:55:29.637 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:29 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-07-14T11:55:29.637 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:29 smithi097 ceph-mon[129313]: osdmap e20: 3 total, 3 up, 3 in 2023-07-14T11:55:29.638 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:29 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-07-14T11:55:29.645 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:29 smithi017 ceph-mon[126339]: pgmap v80: 0 pgs: ; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-14T11:55:29.645 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:29 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-07-14T11:55:29.645 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:29 smithi017 ceph-mon[126339]: from='client.24188 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:55:29.645 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:29 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:55:29.645 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:29 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:55:29.645 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:29 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:55:29.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:29 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-07-14T11:55:29.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:29 smithi017 ceph-mon[126339]: osdmap e20: 3 total, 3 up, 3 in 2023-07-14T11:55:29.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:29 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-07-14T11:55:30.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:30 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-07-14T11:55:30.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:30 smithi093 ceph-mon[128922]: osdmap e21: 3 total, 3 up, 3 in 2023-07-14T11:55:30.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:30 smithi093 ceph-mon[128922]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-14T11:55:30.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:30 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-07-14T11:55:30.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:30 smithi097 ceph-mon[129313]: osdmap e21: 3 total, 3 up, 3 in 2023-07-14T11:55:30.637 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:30 smithi097 ceph-mon[129313]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-14T11:55:30.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:30 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-07-14T11:55:30.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:30 smithi017 ceph-mon[126339]: osdmap e21: 3 total, 3 up, 3 in 2023-07-14T11:55:30.647 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:30 smithi017 ceph-mon[126339]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-14T11:55:31.146 INFO:journalctl@ceph.osd.0.smithi017.stdout:Jul 14 11:55:30 smithi017 sudo[142569]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-14T11:55:31.146 INFO:journalctl@ceph.osd.0.smithi017.stdout:Jul 14 11:55:30 smithi017 sudo[142569]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T11:55:31.146 INFO:journalctl@ceph.osd.0.smithi017.stdout:Jul 14 11:55:30 smithi017 sudo[142569]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T11:55:31.146 INFO:journalctl@ceph.osd.0.smithi017.stdout:Jul 14 11:55:30 smithi017 sudo[142569]: pam_unix(sudo:session): session closed for user root 2023-07-14T11:55:31.612 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:31 smithi093 ceph-mon[128922]: osdmap e22: 3 total, 3 up, 3 in 2023-07-14T11:55:31.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:31 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:31.613 INFO:journalctl@ceph.osd.1.smithi093.stdout:Jul 14 11:55:31 smithi093 sudo[136344]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-14T11:55:31.613 INFO:journalctl@ceph.osd.1.smithi093.stdout:Jul 14 11:55:31 smithi093 sudo[136344]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T11:55:31.614 INFO:journalctl@ceph.osd.1.smithi093.stdout:Jul 14 11:55:31 smithi093 sudo[136344]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T11:55:31.614 INFO:journalctl@ceph.osd.1.smithi093.stdout:Jul 14 11:55:31 smithi093 sudo[136344]: pam_unix(sudo:session): session closed for user root 2023-07-14T11:55:31.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:31 smithi097 ceph-mon[129313]: osdmap e22: 3 total, 3 up, 3 in 2023-07-14T11:55:31.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:31 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:31.646 INFO:journalctl@ceph.osd.0.smithi017.stdout:Jul 14 11:55:31 smithi017 sudo[142607]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-14T11:55:31.646 INFO:journalctl@ceph.osd.0.smithi017.stdout:Jul 14 11:55:31 smithi017 sudo[142607]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T11:55:31.646 INFO:journalctl@ceph.osd.0.smithi017.stdout:Jul 14 11:55:31 smithi017 sudo[142607]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T11:55:31.646 INFO:journalctl@ceph.osd.0.smithi017.stdout:Jul 14 11:55:31 smithi017 sudo[142607]: pam_unix(sudo:session): session closed for user root 2023-07-14T11:55:31.647 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:31 smithi017 ceph-mon[126339]: osdmap e22: 3 total, 3 up, 3 in 2023-07-14T11:55:31.647 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:31 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:32.112 INFO:journalctl@ceph.osd.1.smithi093.stdout:Jul 14 11:55:31 smithi093 sudo[136347]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-14T11:55:32.113 INFO:journalctl@ceph.osd.1.smithi093.stdout:Jul 14 11:55:31 smithi093 sudo[136347]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T11:55:32.113 INFO:journalctl@ceph.osd.1.smithi093.stdout:Jul 14 11:55:31 smithi093 sudo[136347]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T11:55:32.113 INFO:journalctl@ceph.osd.1.smithi093.stdout:Jul 14 11:55:31 smithi093 sudo[136347]: pam_unix(sudo:session): session closed for user root 2023-07-14T11:55:32.249 INFO:journalctl@ceph.osd.2.smithi097.stdout:Jul 14 11:55:32 smithi097 sudo[137341]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-14T11:55:32.249 INFO:journalctl@ceph.osd.2.smithi097.stdout:Jul 14 11:55:32 smithi097 sudo[137341]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T11:55:32.250 INFO:journalctl@ceph.osd.2.smithi097.stdout:Jul 14 11:55:32 smithi097 sudo[137341]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T11:55:32.250 INFO:journalctl@ceph.osd.2.smithi097.stdout:Jul 14 11:55:32 smithi097 sudo[137341]: pam_unix(sudo:session): session closed for user root 2023-07-14T11:55:32.636 INFO:journalctl@ceph.osd.2.smithi097.stdout:Jul 14 11:55:32 smithi097 sudo[137344]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-14T11:55:32.637 INFO:journalctl@ceph.osd.2.smithi097.stdout:Jul 14 11:55:32 smithi097 sudo[137344]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T11:55:32.637 INFO:journalctl@ceph.osd.2.smithi097.stdout:Jul 14 11:55:32 smithi097 sudo[137344]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T11:55:32.637 INFO:journalctl@ceph.osd.2.smithi097.stdout:Jul 14 11:55:32 smithi097 sudo[137344]: pam_unix(sudo:session): session closed for user root 2023-07-14T11:55:33.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:32 smithi017 sudo[142884]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-14T11:55:33.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:32 smithi017 sudo[142884]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T11:55:33.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:32 smithi017 sudo[142884]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T11:55:33.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:32 smithi017 sudo[142884]: pam_unix(sudo:session): session closed for user root 2023-07-14T11:55:33.368 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:33.386+0000 7f519effd700 1 -- 172.21.15.17:0/3729445776 <== mon.2 v2:172.21.15.93:3300/0 7 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f51a40055f0 con 0x7f51a8072f70 2023-07-14T11:55:33.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:33 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/670099734' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "37a1583c-4fb8-412a-a0c8-f83b1ff26a91"}]: dispatch 2023-07-14T11:55:33.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:33 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/670099734' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "37a1583c-4fb8-412a-a0c8-f83b1ff26a91"}]': finished 2023-07-14T11:55:33.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:33 smithi093 ceph-mon[128922]: osdmap e23: 4 total, 3 up, 4 in 2023-07-14T11:55:33.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:33 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T11:55:33.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:33 smithi093 ceph-mon[128922]: pgmap v86: 1 pgs: 1 active+clean; 449 KiB data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-14T11:55:33.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:33 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:33.614 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:33 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/1819308358' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:55:33.614 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:33 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:33.614 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:33 smithi093 ceph-mon[128922]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-14T11:55:33.614 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:33 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/670099734' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "37a1583c-4fb8-412a-a0c8-f83b1ff26a91"}]: dispatch 2023-07-14T11:55:33.614 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:33 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/670099734' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "37a1583c-4fb8-412a-a0c8-f83b1ff26a91"}]': finished 2023-07-14T11:55:33.615 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:33 smithi097 ceph-mon[129313]: osdmap e23: 4 total, 3 up, 4 in 2023-07-14T11:55:33.615 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:33 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T11:55:33.615 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:33 smithi097 ceph-mon[129313]: pgmap v86: 1 pgs: 1 active+clean; 449 KiB data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-14T11:55:33.615 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:33 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:33.615 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:33 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/1819308358' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:55:33.615 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:33 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:33.615 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:33 smithi097 ceph-mon[129313]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-14T11:55:33.616 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:33 smithi097 sudo[137347]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-14T11:55:33.616 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:33 smithi097 sudo[137347]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T11:55:33.616 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:33 smithi097 sudo[137347]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T11:55:33.616 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:33 smithi097 sudo[137347]: pam_unix(sudo:session): session closed for user root 2023-07-14T11:55:33.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:33 smithi017 sudo[143076]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-14T11:55:33.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:33 smithi017 sudo[143076]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T11:55:33.647 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:33 smithi017 sudo[143076]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T11:55:33.647 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:33 smithi017 sudo[143076]: pam_unix(sudo:session): session closed for user root 2023-07-14T11:55:33.647 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:33 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/670099734' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "37a1583c-4fb8-412a-a0c8-f83b1ff26a91"}]: dispatch 2023-07-14T11:55:33.647 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:33 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/670099734' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "37a1583c-4fb8-412a-a0c8-f83b1ff26a91"}]': finished 2023-07-14T11:55:33.647 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:33 smithi017 ceph-mon[126339]: osdmap e23: 4 total, 3 up, 4 in 2023-07-14T11:55:33.647 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:33 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T11:55:33.647 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:33 smithi017 ceph-mon[126339]: pgmap v86: 1 pgs: 1 active+clean; 449 KiB data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-14T11:55:33.648 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:33 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:33.648 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:33 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/1819308358' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:55:33.648 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:33 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:33.648 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:33 smithi017 ceph-mon[126339]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-14T11:55:33.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:33 smithi097 sudo[137350]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-14T11:55:33.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:33 smithi097 sudo[137350]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T11:55:33.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:33 smithi097 sudo[137350]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T11:55:33.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:33 smithi097 sudo[137350]: pam_unix(sudo:session): session closed for user root 2023-07-14T11:55:34.344 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:34 smithi093 sudo[136350]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-14T11:55:34.345 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:34 smithi093 sudo[136350]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T11:55:34.345 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:34 smithi093 sudo[136350]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T11:55:34.345 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:34 smithi093 sudo[136350]: pam_unix(sudo:session): session closed for user root 2023-07-14T11:55:34.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:34 smithi093 sudo[136353]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-14T11:55:34.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:34 smithi093 sudo[136353]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T11:55:34.614 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:34 smithi093 sudo[136353]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T11:55:34.614 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:34 smithi093 sudo[136353]: pam_unix(sudo:session): session closed for user root 2023-07-14T11:55:34.614 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:34 smithi093 ceph-mon[128922]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-14T11:55:34.614 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:34 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T11:55:34.615 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:34 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:55:34.615 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:34 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:55:34.615 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:34 smithi093 ceph-mon[128922]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-14T11:55:34.616 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:34 smithi093 ceph-mon[128922]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-07-14T11:55:34.616 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:34 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T11:55:34.616 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:34 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:55:34.617 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:34 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:55:34.617 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:34 smithi093 ceph-mon[128922]: pgmap v87: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T11:55:34.617 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:34 smithi093 ceph-mon[128922]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-14T11:55:34.617 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:34 smithi093 ceph-mon[128922]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-14T11:55:34.618 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:34 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T11:55:34.618 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:34 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:55:34.618 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:34 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:55:34.618 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:34 smithi093 ceph-mon[128922]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-14T11:55:34.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:34 smithi097 ceph-mon[129313]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-14T11:55:34.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:34 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T11:55:34.637 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:34 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:55:34.637 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:34 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:55:34.637 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:34 smithi097 ceph-mon[129313]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-14T11:55:34.637 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:34 smithi097 ceph-mon[129313]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-07-14T11:55:34.637 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:34 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T11:55:34.637 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:34 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:55:34.637 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:34 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:55:34.638 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:34 smithi097 ceph-mon[129313]: pgmap v87: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T11:55:34.638 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:34 smithi097 ceph-mon[129313]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-14T11:55:34.638 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:34 smithi097 ceph-mon[129313]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-14T11:55:34.638 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:34 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T11:55:34.638 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:34 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:55:34.638 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:34 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:55:34.638 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:34 smithi097 ceph-mon[129313]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-14T11:55:34.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:34 smithi017 ceph-mon[126339]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-14T11:55:34.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:34 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T11:55:34.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:34 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:55:34.647 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:34 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:55:34.647 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:34 smithi017 ceph-mon[126339]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-14T11:55:34.647 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:34 smithi017 ceph-mon[126339]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-07-14T11:55:34.647 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:34 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T11:55:34.647 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:34 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:55:34.647 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:34 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:55:34.648 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:34 smithi017 ceph-mon[126339]: pgmap v87: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T11:55:34.648 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:34 smithi017 ceph-mon[126339]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-14T11:55:34.648 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:34 smithi017 ceph-mon[126339]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-14T11:55:34.648 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:34 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T11:55:34.648 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:34 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T11:55:34.648 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:34 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T11:55:34.648 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:34 smithi017 ceph-mon[126339]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-14T11:55:37.101 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:36 smithi017 ceph-mon[126339]: pgmap v88: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T11:55:37.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:36 smithi093 ceph-mon[128922]: pgmap v88: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T11:55:37.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:36 smithi097 ceph-mon[129313]: pgmap v88: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T11:55:39.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:38 smithi093 ceph-mon[128922]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 6.3 KiB/s rd, 290 KiB/s wr, 19 op/s 2023-07-14T11:55:39.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:38 smithi097 ceph-mon[129313]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 6.3 KiB/s rd, 290 KiB/s wr, 19 op/s 2023-07-14T11:55:39.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:38 smithi017 ceph-mon[126339]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 6.3 KiB/s rd, 290 KiB/s wr, 19 op/s 2023-07-14T11:55:39.974 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:39 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-07-14T11:55:39.975 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:39 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:55:39.975 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:39 smithi017 ceph-mon[126339]: Deploying daemon osd.3 on smithi017 2023-07-14T11:55:40.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:39 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-07-14T11:55:40.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:39 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:55:40.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:39 smithi093 ceph-mon[128922]: Deploying daemon osd.3 on smithi017 2023-07-14T11:55:40.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:39 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-07-14T11:55:40.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:39 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:55:40.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:39 smithi097 ceph-mon[129313]: Deploying daemon osd.3 on smithi017 2023-07-14T11:55:40.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:40 smithi017 ceph-mon[126339]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 4.8 KiB/s rd, 220 KiB/s wr, 15 op/s 2023-07-14T11:55:41.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:40 smithi093 ceph-mon[128922]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 4.8 KiB/s rd, 220 KiB/s wr, 15 op/s 2023-07-14T11:55:41.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:40 smithi097 ceph-mon[129313]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 4.8 KiB/s rd, 220 KiB/s wr, 15 op/s 2023-07-14T11:55:43.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:42 smithi093 ceph-mon[128922]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.9 KiB/s rd, 177 KiB/s wr, 12 op/s 2023-07-14T11:55:43.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:42 smithi097 ceph-mon[129313]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.9 KiB/s rd, 177 KiB/s wr, 12 op/s 2023-07-14T11:55:43.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:42 smithi017 ceph-mon[126339]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.9 KiB/s rd, 177 KiB/s wr, 12 op/s 2023-07-14T11:55:44.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:44 smithi097 ceph-mon[129313]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2023-07-14T11:55:44.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:44 smithi017 ceph-mon[126339]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2023-07-14T11:55:44.862 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:44 smithi093 ceph-mon[128922]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2023-07-14T11:55:46.668 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:46 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:46.668 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:46 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:46.668 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:46 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:55:46.669 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:46 smithi017 ceph-mon[126339]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2023-07-14T11:55:46.862 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:46 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:46.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:46 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:46.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:46 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:55:46.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:46 smithi093 ceph-mon[128922]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2023-07-14T11:55:46.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:46 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:46.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:46 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:46.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:46 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:55:46.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:46 smithi097 ceph-mon[129313]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2023-07-14T11:55:49.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:48 smithi093 ceph-mon[128922]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2023-07-14T11:55:49.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:48 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:49.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:48 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:49.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:48 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:55:49.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:48 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:55:49.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:48 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:49.120 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:48 smithi017 ceph-mon[126339]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2023-07-14T11:55:49.121 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:48 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:49.121 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:48 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:49.121 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:48 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:55:49.121 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:48 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:55:49.121 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:48 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:49.130 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:49.148+0000 7f519effd700 1 -- 172.21.15.17:0/3729445776 <== mgr.14152 v2:172.21.15.17:6800/4237449858 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f51a8060950 con 0x7f51940a10c0 2023-07-14T11:55:49.130 INFO:teuthology.orchestra.run.smithi017.stdout:Created osd(s) 3 on host 'smithi017' 2023-07-14T11:55:49.133 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:49.151+0000 7f51afc35700 1 -- 172.21.15.17:0/3729445776 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f51940a10c0 msgr2=0x7f51940a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:49.133 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:49.151+0000 7f51afc35700 1 --2- 172.21.15.17:0/3729445776 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f51940a10c0 0x7f51940a3570 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5198005950 tx=0x7f519800b410 comp rx=0 tx=0).stop 2023-07-14T11:55:49.133 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:49.151+0000 7f51afc35700 1 -- 172.21.15.17:0/3729445776 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51a8072f70 msgr2=0x7f51a8197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:49.134 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:49.151+0000 7f51afc35700 1 --2- 172.21.15.17:0/3729445776 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51a8072f70 0x7f51a8197ac0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f51a4004730 tx=0x7f51a4004760 comp rx=0 tx=0).stop 2023-07-14T11:55:49.134 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:49.151+0000 7f51afc35700 1 -- 172.21.15.17:0/3729445776 shutdown_connections 2023-07-14T11:55:49.134 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:49.151+0000 7f51afc35700 1 --2- 172.21.15.17:0/3729445776 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f51940a10c0 0x7f51940a3570 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:49.134 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:49.151+0000 7f51afc35700 1 --2- 172.21.15.17:0/3729445776 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51a8072f70 0x7f51a8197ac0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:49.134 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:49.151+0000 7f51afc35700 1 --2- 172.21.15.17:0/3729445776 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51a8107a60 0x7f51a819f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:49.134 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:49.151+0000 7f51afc35700 1 --2- 172.21.15.17:0/3729445776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51a8074540 0x7f51a8198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:49.134 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:49.151+0000 7f51afc35700 1 -- 172.21.15.17:0/3729445776 >> 172.21.15.17:0/3729445776 conn(0x7f51a80fb1a0 msgr2=0x7f51a80fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:49.135 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:49.151+0000 7f51afc35700 1 -- 172.21.15.17:0/3729445776 shutdown_connections 2023-07-14T11:55:49.135 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:55:49.151+0000 7f51afc35700 1 -- 172.21.15.17:0/3729445776 wait complete. 2023-07-14T11:55:49.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:48 smithi097 ceph-mon[129313]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2023-07-14T11:55:49.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:48 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:49.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:48 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:49.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:48 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:55:49.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:48 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:55:49.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:48 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:49.741 DEBUG:teuthology.orchestra.run.smithi017:osd.3> sudo journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@osd.3.service 2023-07-14T11:55:49.746 INFO:tasks.cephadm:Deploying osd.4 on smithi093 with /dev/vg_nvme/lv_3... 2023-07-14T11:55:49.746 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-07-14T11:55:49.819 INFO:journalctl@ceph.osd.3.smithi017.stdout:-- Logs begin at Fri 2023-07-14 11:32:03 UTC. -- 2023-07-14T11:55:50.310 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.b/config 2023-07-14T11:55:50.354 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:50 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:50.354 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:50 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:50.354 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:50 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:55:50.355 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:50 smithi017 ceph-mon[126339]: from='osd.3 [v2:172.21.15.17:6810/2784619821,v1:172.21.15.17:6811/2784619821]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-07-14T11:55:50.355 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:50 smithi017 ceph-mon[126339]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T11:55:50.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:50 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:50.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:50 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:50.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:50 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:55:50.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:50 smithi093 ceph-mon[128922]: from='osd.3 [v2:172.21.15.17:6810/2784619821,v1:172.21.15.17:6811/2784619821]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-07-14T11:55:50.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:50 smithi093 ceph-mon[128922]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T11:55:50.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:50 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:50.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:50 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:50.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:50 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:55:50.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:50 smithi097 ceph-mon[129313]: from='osd.3 [v2:172.21.15.17:6810/2784619821,v1:172.21.15.17:6811/2784619821]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-07-14T11:55:50.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:50 smithi097 ceph-mon[129313]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T11:55:51.306 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:51 smithi017 ceph-mon[126339]: from='osd.3 [v2:172.21.15.17:6810/2784619821,v1:172.21.15.17:6811/2784619821]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-07-14T11:55:51.306 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:51 smithi017 ceph-mon[126339]: osdmap e24: 4 total, 3 up, 4 in 2023-07-14T11:55:51.306 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:51 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T11:55:51.306 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:51 smithi017 ceph-mon[126339]: from='osd.3 [v2:172.21.15.17:6810/2784619821,v1:172.21.15.17:6811/2784619821]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2023-07-14T11:55:51.307 INFO:journalctl@ceph.osd.3.smithi017.stdout:Jul 14 11:55:51 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-3[144885]: 2023-07-14T11:55:51.133+0000 7f626c6f1700 -1 osd.3 0 waiting for initial osdmap 2023-07-14T11:55:51.307 INFO:journalctl@ceph.osd.3.smithi017.stdout:Jul 14 11:55:51 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-3[144885]: 2023-07-14T11:55:51.141+0000 7f6266ce3700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T11:55:51.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:51 smithi097 ceph-mon[129313]: from='osd.3 [v2:172.21.15.17:6810/2784619821,v1:172.21.15.17:6811/2784619821]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-07-14T11:55:51.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:51 smithi097 ceph-mon[129313]: osdmap e24: 4 total, 3 up, 4 in 2023-07-14T11:55:51.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:51 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T11:55:51.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:51 smithi097 ceph-mon[129313]: from='osd.3 [v2:172.21.15.17:6810/2784619821,v1:172.21.15.17:6811/2784619821]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2023-07-14T11:55:51.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:51 smithi093 ceph-mon[128922]: from='osd.3 [v2:172.21.15.17:6810/2784619821,v1:172.21.15.17:6811/2784619821]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-07-14T11:55:51.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:51 smithi093 ceph-mon[128922]: osdmap e24: 4 total, 3 up, 4 in 2023-07-14T11:55:51.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:51 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T11:55:51.614 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:51 smithi093 ceph-mon[128922]: from='osd.3 [v2:172.21.15.17:6810/2784619821,v1:172.21.15.17:6811/2784619821]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2023-07-14T11:55:51.684 INFO:teuthology.orchestra.run.smithi093.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-07-14T11:55:51.684 INFO:teuthology.orchestra.run.smithi093.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-07-14T11:55:51.684 INFO:teuthology.orchestra.run.smithi093.stderr: stderr: 10+0 records in 2023-07-14T11:55:51.685 INFO:teuthology.orchestra.run.smithi093.stderr:10+0 records out 2023-07-14T11:55:51.685 INFO:teuthology.orchestra.run.smithi093.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0109382 s, 959 MB/s 2023-07-14T11:55:51.685 INFO:teuthology.orchestra.run.smithi093.stderr:--> Zapping successful for: 2023-07-14T11:55:52.061 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi093:vg_nvme/lv_3 2023-07-14T11:55:52.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:52 smithi093 ceph-mon[128922]: from='osd.3 [v2:172.21.15.17:6810/2784619821,v1:172.21.15.17:6811/2784619821]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2023-07-14T11:55:52.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:52 smithi093 ceph-mon[128922]: osdmap e25: 4 total, 3 up, 4 in 2023-07-14T11:55:52.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:52 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T11:55:52.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:52 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T11:55:52.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:52 smithi093 ceph-mon[128922]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T11:55:52.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:52 smithi097 ceph-mon[129313]: from='osd.3 [v2:172.21.15.17:6810/2784619821,v1:172.21.15.17:6811/2784619821]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2023-07-14T11:55:52.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:52 smithi097 ceph-mon[129313]: osdmap e25: 4 total, 3 up, 4 in 2023-07-14T11:55:52.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:52 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T11:55:52.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:52 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T11:55:52.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:52 smithi097 ceph-mon[129313]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T11:55:52.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:52 smithi017 ceph-mon[126339]: from='osd.3 [v2:172.21.15.17:6810/2784619821,v1:172.21.15.17:6811/2784619821]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2023-07-14T11:55:52.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:52 smithi017 ceph-mon[126339]: osdmap e25: 4 total, 3 up, 4 in 2023-07-14T11:55:52.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:52 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T11:55:52.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:52 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T11:55:52.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:52 smithi017 ceph-mon[126339]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T11:55:52.616 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.b/config 2023-07-14T11:55:53.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:53 smithi097 ceph-mon[129313]: purged_snaps scrub starts 2023-07-14T11:55:53.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:53 smithi097 ceph-mon[129313]: purged_snaps scrub ok 2023-07-14T11:55:53.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:53 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T11:55:53.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:53 smithi097 ceph-mon[129313]: osd.3 [v2:172.21.15.17:6810/2784619821,v1:172.21.15.17:6811/2784619821] boot 2023-07-14T11:55:53.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:53 smithi097 ceph-mon[129313]: osdmap e26: 4 total, 4 up, 4 in 2023-07-14T11:55:53.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:53 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T11:55:53.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:53 smithi097 ceph-mon[129313]: Detected new or changed devices on smithi017 2023-07-14T11:55:53.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:53 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:53.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:53 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:53.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:53 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:55:53.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:53 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:55:53.388 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:53 smithi097 ceph-mon[129313]: Adjusting osd_memory_target on smithi017 to 8636M 2023-07-14T11:55:53.388 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:53 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:53.388 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:53 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:55:53.388 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:53 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:55:53.388 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:53 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:53.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:53 smithi017 ceph-mon[126339]: purged_snaps scrub starts 2023-07-14T11:55:53.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:53 smithi017 ceph-mon[126339]: purged_snaps scrub ok 2023-07-14T11:55:53.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:53 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T11:55:53.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:53 smithi017 ceph-mon[126339]: osd.3 [v2:172.21.15.17:6810/2784619821,v1:172.21.15.17:6811/2784619821] boot 2023-07-14T11:55:53.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:53 smithi017 ceph-mon[126339]: osdmap e26: 4 total, 4 up, 4 in 2023-07-14T11:55:53.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:53 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T11:55:53.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:53 smithi017 ceph-mon[126339]: Detected new or changed devices on smithi017 2023-07-14T11:55:53.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:53 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:53.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:53 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:53.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:53 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:55:53.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:53 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:55:53.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:53 smithi017 ceph-mon[126339]: Adjusting osd_memory_target on smithi017 to 8636M 2023-07-14T11:55:53.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:53 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:53.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:53 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:55:53.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:53 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:55:53.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:53 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:53.612 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:53 smithi093 ceph-mon[128922]: purged_snaps scrub starts 2023-07-14T11:55:53.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:53 smithi093 ceph-mon[128922]: purged_snaps scrub ok 2023-07-14T11:55:53.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:53 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T11:55:53.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:53 smithi093 ceph-mon[128922]: osd.3 [v2:172.21.15.17:6810/2784619821,v1:172.21.15.17:6811/2784619821] boot 2023-07-14T11:55:53.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:53 smithi093 ceph-mon[128922]: osdmap e26: 4 total, 4 up, 4 in 2023-07-14T11:55:53.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:53 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T11:55:53.614 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:53 smithi093 ceph-mon[128922]: Detected new or changed devices on smithi017 2023-07-14T11:55:53.614 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:53 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:53.614 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:53 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:53.614 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:53 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:55:53.614 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:53 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:55:53.614 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:53 smithi093 ceph-mon[128922]: Adjusting osd_memory_target on smithi017 to 8636M 2023-07-14T11:55:53.615 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:53 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:53.615 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:53 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:55:53.615 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:53 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:55:53.615 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:53 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:55:53.789 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:53.807+0000 7f0da69ff700 1 -- 172.21.15.93:0/1219415420 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0da01006f0 msgr2=0x7f0da0106f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:53.789 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:53.807+0000 7f0da69ff700 1 --2- 172.21.15.93:0/1219415420 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0da01006f0 0x7f0da0106f90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0d9800b0a0 tx=0x7f0d9800b3b0 comp rx=0 tx=0).stop 2023-07-14T11:55:53.789 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:53.807+0000 7f0da69ff700 1 -- 172.21.15.93:0/1219415420 shutdown_connections 2023-07-14T11:55:53.789 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:53.808+0000 7f0da69ff700 1 --2- 172.21.15.93:0/1219415420 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0da01006f0 0x7f0da0106f90 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:53.790 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:53.808+0000 7f0da69ff700 1 --2- 172.21.15.93:0/1219415420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0da00feaa0 0x7f0da00feeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:53.790 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:53.808+0000 7f0da69ff700 1 --2- 172.21.15.93:0/1219415420 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0da00ffd40 0x7f0da01001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:53.790 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:53.808+0000 7f0da69ff700 1 -- 172.21.15.93:0/1219415420 >> 172.21.15.93:0/1219415420 conn(0x7f0da00fa110 msgr2=0x7f0da00fc520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:55:53.790 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:53.808+0000 7f0da69ff700 1 -- 172.21.15.93:0/1219415420 shutdown_connections 2023-07-14T11:55:53.790 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:53.808+0000 7f0da69ff700 1 -- 172.21.15.93:0/1219415420 wait complete. 2023-07-14T11:55:53.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:53.808+0000 7f0da69ff700 1 Processor -- start 2023-07-14T11:55:53.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:53.808+0000 7f0da69ff700 1 -- start start 2023-07-14T11:55:53.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:53.809+0000 7f0da69ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0da00feaa0 0x7f0da0102d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:53.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:53.809+0000 7f0da69ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0da00ffd40 0x7f0da0101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:53.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:53.809+0000 7f0da69ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0da01006f0 0x7f0da01018d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:53.793 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:53.809+0000 7f0da69ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da0102050 con 0x7f0da00ffd40 2023-07-14T11:55:53.793 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:53.809+0000 7f0da69ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da01021c0 con 0x7f0da01006f0 2023-07-14T11:55:53.793 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:53.809+0000 7f0da69ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da019f9f0 con 0x7f0da00feaa0 2023-07-14T11:55:53.793 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:53.809+0000 7f0da4f9c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0da01006f0 0x7f0da01018d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:53.793 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:53.809+0000 7f0da4f9c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0da01006f0 0x7f0da01018d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:57678/0 (socket says 172.21.15.93:57678) 2023-07-14T11:55:53.793 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:53.809+0000 7f0da4f9c700 1 -- 172.21.15.93:0/1112920232 learned_addr learned my addr 172.21.15.93:0/1112920232 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-14T11:55:53.793 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:53.809+0000 7f0d9ffff700 1 --2- 172.21.15.93:0/1112920232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0da00feaa0 0x7f0da0102d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:53.794 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:53.809+0000 7f0d9f7fe700 1 --2- 172.21.15.93:0/1112920232 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0da00ffd40 0x7f0da0101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:53.794 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:53.809+0000 7f0da4f9c700 1 -- 172.21.15.93:0/1112920232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0da00feaa0 msgr2=0x7f0da0102d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:53.794 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:53.809+0000 7f0da4f9c700 1 --2- 172.21.15.93:0/1112920232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0da00feaa0 0x7f0da0102d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:53.794 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:53.809+0000 7f0da4f9c700 1 -- 172.21.15.93:0/1112920232 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0da00ffd40 msgr2=0x7f0da0101390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:55:53.794 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:53.809+0000 7f0da4f9c700 1 --2- 172.21.15.93:0/1112920232 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0da00ffd40 0x7f0da0101390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:55:53.794 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:53.809+0000 7f0da4f9c700 1 -- 172.21.15.93:0/1112920232 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d98009d00 con 0x7f0da01006f0 2023-07-14T11:55:53.794 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:53.809+0000 7f0da4f9c700 1 --2- 172.21.15.93:0/1112920232 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0da01006f0 0x7f0da01018d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0d980062a0 tx=0x7f0d98004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:53.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:53.809+0000 7f0d9d7fa700 1 -- 172.21.15.93:0/1112920232 <== mon.2 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9800e040 con 0x7f0da01006f0 2023-07-14T11:55:53.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:53.809+0000 7f0d9d7fa700 1 -- 172.21.15.93:0/1112920232 <== mon.2 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d9800cec0 con 0x7f0da01006f0 2023-07-14T11:55:53.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:53.809+0000 7f0da69ff700 1 -- 172.21.15.93:0/1112920232 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0da019fc70 con 0x7f0da01006f0 2023-07-14T11:55:53.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:53.809+0000 7f0d9d7fa700 1 -- 172.21.15.93:0/1112920232 <== mon.2 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d98004020 con 0x7f0da01006f0 2023-07-14T11:55:53.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:53.809+0000 7f0da69ff700 1 -- 172.21.15.93:0/1112920232 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0da01a0090 con 0x7f0da01006f0 2023-07-14T11:55:53.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:53.810+0000 7f0da69ff700 1 -- 172.21.15.93:0/1112920232 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0da0106b90 con 0x7f0da01006f0 2023-07-14T11:55:53.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:53.811+0000 7f0d9d7fa700 1 -- 172.21.15.93:0/1112920232 <== mon.2 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f0d9801d690 con 0x7f0da01006f0 2023-07-14T11:55:53.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:53.812+0000 7f0d9d7fa700 1 --2- 172.21.15.93:0/1112920232 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f0d880a1150 0x7f0d880a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:55:53.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:53.812+0000 7f0d9d7fa700 1 -- 172.21.15.93:0/1112920232 <== mon.2 v2:172.21.15.93:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 3698+0+0 (secure 0 0 0) 0x7f0d980cda10 con 0x7f0da01006f0 2023-07-14T11:55:53.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:53.812+0000 7f0d9ffff700 1 --2- 172.21.15.93:0/1112920232 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f0d880a1150 0x7f0d880a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:55:53.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:53.813+0000 7f0d9ffff700 1 --2- 172.21.15.93:0/1112920232 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f0d880a1150 0x7f0d880a3600 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0d90007c60 tx=0x7f0d900073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:55:53.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:53.816+0000 7f0d9d7fa700 1 -- 172.21.15.93:0/1112920232 <== mon.2 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0d980983f0 con 0x7f0da01006f0 2023-07-14T11:55:53.987 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:55:54.006+0000 7f0da69ff700 1 -- 172.21.15.93:0/1112920232 --> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f0da0060950 con 0x7f0d880a1150 2023-07-14T11:55:54.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:54 smithi017 ceph-mon[126339]: osdmap e27: 4 total, 4 up, 4 in 2023-07-14T11:55:54.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:54 smithi017 ceph-mon[126339]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T11:55:54.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:54 smithi017 ceph-mon[126339]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:55:54.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:54 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:55:54.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:54 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:55:54.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:54 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:55:54.612 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:54 smithi093 ceph-mon[128922]: osdmap e27: 4 total, 4 up, 4 in 2023-07-14T11:55:54.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:54 smithi093 ceph-mon[128922]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T11:55:54.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:54 smithi093 ceph-mon[128922]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:55:54.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:54 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:55:54.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:54 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:55:54.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:54 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:55:54.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:54 smithi097 ceph-mon[129313]: osdmap e27: 4 total, 4 up, 4 in 2023-07-14T11:55:54.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:54 smithi097 ceph-mon[129313]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T11:55:54.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:54 smithi097 ceph-mon[129313]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:55:54.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:54 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:55:54.637 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:54 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:55:54.637 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:54 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:55:55.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:55 smithi017 ceph-mon[126339]: osdmap e28: 4 total, 4 up, 4 in 2023-07-14T11:55:55.599 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:55 smithi093 ceph-mon[128922]: osdmap e28: 4 total, 4 up, 4 in 2023-07-14T11:55:55.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:55 smithi097 ceph-mon[129313]: osdmap e28: 4 total, 4 up, 4 in 2023-07-14T11:55:56.612 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:56 smithi093 ceph-mon[128922]: osdmap e29: 4 total, 4 up, 4 in 2023-07-14T11:55:56.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:56 smithi093 ceph-mon[128922]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T11:55:56.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:56 smithi097 ceph-mon[129313]: osdmap e29: 4 total, 4 up, 4 in 2023-07-14T11:55:56.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:56 smithi097 ceph-mon[129313]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T11:55:56.645 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:56 smithi017 ceph-mon[126339]: osdmap e29: 4 total, 4 up, 4 in 2023-07-14T11:55:56.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:56 smithi017 ceph-mon[126339]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T11:55:57.612 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:57 smithi093 ceph-mon[128922]: osdmap e30: 4 total, 4 up, 4 in 2023-07-14T11:55:57.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:57 smithi093 ceph-mon[128922]: from='client.? 172.21.15.93:0/1535821668' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4b9f377-d519-4b14-ac4a-f1debaeed9f0"}]: dispatch 2023-07-14T11:55:57.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:57 smithi093 ceph-mon[128922]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4b9f377-d519-4b14-ac4a-f1debaeed9f0"}]: dispatch 2023-07-14T11:55:57.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:57 smithi093 ceph-mon[128922]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f4b9f377-d519-4b14-ac4a-f1debaeed9f0"}]': finished 2023-07-14T11:55:57.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:57 smithi093 ceph-mon[128922]: osdmap e31: 5 total, 4 up, 5 in 2023-07-14T11:55:57.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:57 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T11:55:57.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:57 smithi093 ceph-mon[128922]: from='client.? 172.21.15.93:0/3650479331' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:55:57.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:57 smithi097 ceph-mon[129313]: osdmap e30: 4 total, 4 up, 4 in 2023-07-14T11:55:57.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:57 smithi097 ceph-mon[129313]: from='client.? 172.21.15.93:0/1535821668' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4b9f377-d519-4b14-ac4a-f1debaeed9f0"}]: dispatch 2023-07-14T11:55:57.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:57 smithi097 ceph-mon[129313]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4b9f377-d519-4b14-ac4a-f1debaeed9f0"}]: dispatch 2023-07-14T11:55:57.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:57 smithi097 ceph-mon[129313]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f4b9f377-d519-4b14-ac4a-f1debaeed9f0"}]': finished 2023-07-14T11:55:57.637 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:57 smithi097 ceph-mon[129313]: osdmap e31: 5 total, 4 up, 5 in 2023-07-14T11:55:57.637 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:57 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T11:55:57.637 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:57 smithi097 ceph-mon[129313]: from='client.? 172.21.15.93:0/3650479331' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:55:57.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:57 smithi017 ceph-mon[126339]: osdmap e30: 4 total, 4 up, 4 in 2023-07-14T11:55:57.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:57 smithi017 ceph-mon[126339]: from='client.? 172.21.15.93:0/1535821668' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4b9f377-d519-4b14-ac4a-f1debaeed9f0"}]: dispatch 2023-07-14T11:55:57.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:57 smithi017 ceph-mon[126339]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4b9f377-d519-4b14-ac4a-f1debaeed9f0"}]: dispatch 2023-07-14T11:55:57.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:57 smithi017 ceph-mon[126339]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f4b9f377-d519-4b14-ac4a-f1debaeed9f0"}]': finished 2023-07-14T11:55:57.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:57 smithi017 ceph-mon[126339]: osdmap e31: 5 total, 4 up, 5 in 2023-07-14T11:55:57.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:57 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T11:55:57.647 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:57 smithi017 ceph-mon[126339]: from='client.? 172.21.15.93:0/3650479331' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:55:58.612 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:55:58 smithi093 ceph-mon[128922]: pgmap v107: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-14T11:55:58.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:55:58 smithi097 ceph-mon[129313]: pgmap v107: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-14T11:55:58.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:55:58 smithi017 ceph-mon[126339]: pgmap v107: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-14T11:56:01.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:00 smithi093 ceph-mon[128922]: pgmap v108: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 104 KiB/s, 0 objects/s recovering 2023-07-14T11:56:01.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:00 smithi097 ceph-mon[129313]: pgmap v108: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 104 KiB/s, 0 objects/s recovering 2023-07-14T11:56:01.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:00 smithi017 ceph-mon[126339]: pgmap v108: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 104 KiB/s, 0 objects/s recovering 2023-07-14T11:56:03.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:02 smithi093 ceph-mon[128922]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 88 KiB/s, 0 objects/s recovering 2023-07-14T11:56:03.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:02 smithi097 ceph-mon[129313]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 88 KiB/s, 0 objects/s recovering 2023-07-14T11:56:03.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:02 smithi017 ceph-mon[126339]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 88 KiB/s, 0 objects/s recovering 2023-07-14T11:56:04.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:03 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-07-14T11:56:04.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:03 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:56:04.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:03 smithi093 ceph-mon[128922]: Deploying daemon osd.4 on smithi093 2023-07-14T11:56:04.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:03 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-07-14T11:56:04.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:03 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:56:04.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:03 smithi097 ceph-mon[129313]: Deploying daemon osd.4 on smithi093 2023-07-14T11:56:04.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:03 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-07-14T11:56:04.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:03 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:56:04.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:03 smithi017 ceph-mon[126339]: Deploying daemon osd.4 on smithi093 2023-07-14T11:56:04.862 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:04 smithi093 ceph-mon[128922]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-14T11:56:05.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:04 smithi097 ceph-mon[129313]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-14T11:56:05.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:04 smithi017 ceph-mon[126339]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-14T11:56:07.014 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:06 smithi093 ceph-mon[128922]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T11:56:07.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:06 smithi097 ceph-mon[129313]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T11:56:07.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:06 smithi017 ceph-mon[126339]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T11:56:08.308 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:08 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:08.308 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:08 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:56:08.308 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:08 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:08.308 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:08 smithi093 ceph-mon[128922]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T11:56:08.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:08 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:08.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:08 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:56:08.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:08 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:08.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:08 smithi097 ceph-mon[129313]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T11:56:08.645 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:08 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:08.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:08 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:56:08.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:08 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:08.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:08 smithi017 ceph-mon[126339]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T11:56:10.052 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:56:10.070+0000 7f0d9d7fa700 1 -- 172.21.15.93:0/1112920232 <== mgr.14152 v2:172.21.15.17:6800/4237449858 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0da0060950 con 0x7f0d880a1150 2023-07-14T11:56:10.053 INFO:teuthology.orchestra.run.smithi093.stdout:Created osd(s) 4 on host 'smithi093' 2023-07-14T11:56:10.055 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:56:10.072+0000 7f0da69ff700 1 -- 172.21.15.93:0/1112920232 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f0d880a1150 msgr2=0x7f0d880a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:10.055 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:56:10.072+0000 7f0da69ff700 1 --2- 172.21.15.93:0/1112920232 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f0d880a1150 0x7f0d880a3600 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0d90007c60 tx=0x7f0d900073d0 comp rx=0 tx=0).stop 2023-07-14T11:56:10.055 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:56:10.072+0000 7f0da69ff700 1 -- 172.21.15.93:0/1112920232 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0da01006f0 msgr2=0x7f0da01018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:10.055 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:56:10.072+0000 7f0da69ff700 1 --2- 172.21.15.93:0/1112920232 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0da01006f0 0x7f0da01018d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0d980062a0 tx=0x7f0d98004d00 comp rx=0 tx=0).stop 2023-07-14T11:56:10.055 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:56:10.072+0000 7f0da69ff700 1 -- 172.21.15.93:0/1112920232 shutdown_connections 2023-07-14T11:56:10.055 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:56:10.072+0000 7f0da69ff700 1 --2- 172.21.15.93:0/1112920232 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f0d880a1150 0x7f0d880a3600 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:10.056 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:56:10.072+0000 7f0da69ff700 1 --2- 172.21.15.93:0/1112920232 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0da01006f0 0x7f0da01018d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:10.056 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:56:10.072+0000 7f0da69ff700 1 --2- 172.21.15.93:0/1112920232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0da00feaa0 0x7f0da0102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:10.056 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:56:10.072+0000 7f0da69ff700 1 --2- 172.21.15.93:0/1112920232 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0da00ffd40 0x7f0da0101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:10.056 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:56:10.072+0000 7f0da69ff700 1 -- 172.21.15.93:0/1112920232 >> 172.21.15.93:0/1112920232 conn(0x7f0da00fa110 msgr2=0x7f0da00fc280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:10.056 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:56:10.072+0000 7f0da69ff700 1 -- 172.21.15.93:0/1112920232 shutdown_connections 2023-07-14T11:56:10.056 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:56:10.072+0000 7f0da69ff700 1 -- 172.21.15.93:0/1112920232 wait complete. 2023-07-14T11:56:10.509 DEBUG:teuthology.orchestra.run.smithi093:osd.4> sudo journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@osd.4.service 2023-07-14T11:56:10.511 INFO:tasks.cephadm:Deploying osd.5 on smithi097 with /dev/vg_nvme/lv_3... 2023-07-14T11:56:10.511 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-07-14T11:56:10.551 INFO:journalctl@ceph.osd.4.smithi093.stdout:-- Logs begin at Fri 2023-07-14 11:24:16 UTC. -- 2023-07-14T11:56:10.862 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:10 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:10.862 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:10 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:10.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:10 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:56:10.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:10 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:56:10.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:10 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:10.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:10 smithi093 ceph-mon[128922]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T11:56:10.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:10 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:10.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:10 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:56:10.864 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:10 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:10.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:10 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:10.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:10 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:10.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:10 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:56:10.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:10 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:56:10.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:10 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:10.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:10 smithi097 ceph-mon[129313]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T11:56:10.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:10 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:10.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:10 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:56:10.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:10 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:10.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:10 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:10.896 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:10 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:10.896 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:10 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:56:10.896 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:10 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:56:10.896 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:10 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:10.896 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:10 smithi017 ceph-mon[126339]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T11:56:10.896 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:10 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:10.896 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:10 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:56:10.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:10 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:11.085 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T11:56:11.233 INFO:journalctl@ceph.osd.4.smithi093.stdout:Jul 14 11:56:11 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-4[139553]: 2023-07-14T11:56:11.171+0000 7efe64ecb640 -1 osd.4 0 log_to_monitors true 2023-07-14T11:56:11.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:11 smithi097 ceph-mon[129313]: from='osd.4 [v2:172.21.15.93:6808/3151801173,v1:172.21.15.93:6809/3151801173]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-14T11:56:11.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:11 smithi097 ceph-mon[129313]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-14T11:56:11.896 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:11 smithi017 ceph-mon[126339]: from='osd.4 [v2:172.21.15.93:6808/3151801173,v1:172.21.15.93:6809/3151801173]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-14T11:56:11.896 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:11 smithi017 ceph-mon[126339]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-14T11:56:12.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:11 smithi093 ceph-mon[128922]: from='osd.4 [v2:172.21.15.93:6808/3151801173,v1:172.21.15.93:6809/3151801173]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-14T11:56:12.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:11 smithi093 ceph-mon[128922]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-14T11:56:12.566 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-07-14T11:56:12.567 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-07-14T11:56:12.567 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2023-07-14T11:56:12.567 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2023-07-14T11:56:12.568 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0126591 s, 828 MB/s 2023-07-14T11:56:12.568 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2023-07-14T11:56:12.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:12 smithi097 ceph-mon[129313]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-07-14T11:56:12.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:12 smithi097 ceph-mon[129313]: osdmap e32: 5 total, 4 up, 5 in 2023-07-14T11:56:12.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:12 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T11:56:12.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:12 smithi097 ceph-mon[129313]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-07-14T11:56:12.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:12 smithi097 ceph-mon[129313]: from='osd.4 [v2:172.21.15.93:6808/3151801173,v1:172.21.15.93:6809/3151801173]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-07-14T11:56:12.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:12 smithi097 ceph-mon[129313]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T11:56:12.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:12 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:12.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:12 smithi017 ceph-mon[126339]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-07-14T11:56:12.896 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:12 smithi017 ceph-mon[126339]: osdmap e32: 5 total, 4 up, 5 in 2023-07-14T11:56:12.896 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:12 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T11:56:12.896 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:12 smithi017 ceph-mon[126339]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-07-14T11:56:12.896 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:12 smithi017 ceph-mon[126339]: from='osd.4 [v2:172.21.15.93:6808/3151801173,v1:172.21.15.93:6809/3151801173]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-07-14T11:56:12.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:12 smithi017 ceph-mon[126339]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T11:56:12.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:12 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:12.937 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi097:vg_nvme/lv_3 2023-07-14T11:56:13.112 INFO:journalctl@ceph.osd.4.smithi093.stdout:Jul 14 11:56:12 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-4[139553]: 2023-07-14T11:56:12.642+0000 7efe5a543700 -1 osd.4 0 waiting for initial osdmap 2023-07-14T11:56:13.113 INFO:journalctl@ceph.osd.4.smithi093.stdout:Jul 14 11:56:12 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-4[139553]: 2023-07-14T11:56:12.648+0000 7efe54334700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T11:56:13.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:12 smithi093 ceph-mon[128922]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-07-14T11:56:13.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:12 smithi093 ceph-mon[128922]: osdmap e32: 5 total, 4 up, 5 in 2023-07-14T11:56:13.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:12 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T11:56:13.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:12 smithi093 ceph-mon[128922]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-07-14T11:56:13.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:12 smithi093 ceph-mon[128922]: from='osd.4 [v2:172.21.15.93:6808/3151801173,v1:172.21.15.93:6809/3151801173]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-07-14T11:56:13.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:12 smithi093 ceph-mon[128922]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T11:56:13.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:12 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:13.487 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T11:56:13.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:13 smithi097 ceph-mon[129313]: Detected new or changed devices on smithi093 2023-07-14T11:56:13.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:13 smithi097 ceph-mon[129313]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-07-14T11:56:13.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:13 smithi097 ceph-mon[129313]: osdmap e33: 5 total, 4 up, 5 in 2023-07-14T11:56:13.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:13 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T11:56:13.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:13 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T11:56:13.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:13 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:13.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:13 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:56:13.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:13 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:56:13.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:13 smithi097 ceph-mon[129313]: Adjusting osd_memory_target on smithi093 to 8584M 2023-07-14T11:56:13.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:13 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:13.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:13 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:56:13.888 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:13 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:56:13.888 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:13 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:13.896 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:13 smithi017 ceph-mon[126339]: Detected new or changed devices on smithi093 2023-07-14T11:56:13.896 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:13 smithi017 ceph-mon[126339]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-07-14T11:56:13.896 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:13 smithi017 ceph-mon[126339]: osdmap e33: 5 total, 4 up, 5 in 2023-07-14T11:56:13.896 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:13 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T11:56:13.896 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:13 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T11:56:13.896 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:13 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:13.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:13 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:56:13.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:13 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:56:13.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:13 smithi017 ceph-mon[126339]: Adjusting osd_memory_target on smithi093 to 8584M 2023-07-14T11:56:13.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:13 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:13.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:13 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:56:13.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:13 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:56:13.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:13 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:14.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:13 smithi093 ceph-mon[128922]: Detected new or changed devices on smithi093 2023-07-14T11:56:14.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:13 smithi093 ceph-mon[128922]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-07-14T11:56:14.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:13 smithi093 ceph-mon[128922]: osdmap e33: 5 total, 4 up, 5 in 2023-07-14T11:56:14.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:13 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T11:56:14.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:13 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T11:56:14.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:13 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:14.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:13 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:56:14.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:13 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:56:14.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:13 smithi093 ceph-mon[128922]: Adjusting osd_memory_target on smithi093 to 8584M 2023-07-14T11:56:14.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:13 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:14.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:13 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:56:14.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:13 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:56:14.115 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:13 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:14.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:14.723+0000 7fabbebe9700 1 -- 172.21.15.97:0/2651091402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabb80ffba0 msgr2=0x7fabb80fffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:14.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:14.723+0000 7fabbebe9700 1 --2- 172.21.15.97:0/2651091402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabb80ffba0 0x7fabb80fffb0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fabac009990 tx=0x7fabac009ca0 comp rx=0 tx=0).stop 2023-07-14T11:56:14.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:14.723+0000 7fabbebe9700 1 -- 172.21.15.97:0/2651091402 shutdown_connections 2023-07-14T11:56:14.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:14.723+0000 7fabbebe9700 1 --2- 172.21.15.97:0/2651091402 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabb81017f0 0x7fabb8108090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:14.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:14.723+0000 7fabbebe9700 1 --2- 172.21.15.97:0/2651091402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabb80ffba0 0x7fabb80fffb0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:14.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:14.723+0000 7fabbebe9700 1 --2- 172.21.15.97:0/2651091402 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fabb8100e40 0x7fabb81012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:14.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:14.723+0000 7fabbebe9700 1 -- 172.21.15.97:0/2651091402 >> 172.21.15.97:0/2651091402 conn(0x7fabb80fb1f0 msgr2=0x7fabb80fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:14.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:14.723+0000 7fabbebe9700 1 -- 172.21.15.97:0/2651091402 shutdown_connections 2023-07-14T11:56:14.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:14.723+0000 7fabbebe9700 1 -- 172.21.15.97:0/2651091402 wait complete. 2023-07-14T11:56:14.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:14.724+0000 7fabbebe9700 1 Processor -- start 2023-07-14T11:56:14.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:14.724+0000 7fabbebe9700 1 -- start start 2023-07-14T11:56:14.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:14.724+0000 7fabbebe9700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fabb80ffba0 0x7fabb80742b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:14.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:14.724+0000 7fabbebe9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabb8100e40 0x7fabb8072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:14.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:14.724+0000 7fabbebe9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabb81017f0 0x7fabb8072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:14.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:14.724+0000 7fabbebe9700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb80735c0 con 0x7fabb8100e40 2023-07-14T11:56:14.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:14.724+0000 7fabbebe9700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb8073730 con 0x7fabb80ffba0 2023-07-14T11:56:14.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:14.724+0000 7fabbebe9700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb819f930 con 0x7fabb81017f0 2023-07-14T11:56:14.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:14.724+0000 7fabbd186700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabb81017f0 0x7fabb8072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:14.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:14.724+0000 7fabbd186700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabb81017f0 0x7fabb8072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42300/0 (socket says 172.21.15.97:42300) 2023-07-14T11:56:14.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:14.724+0000 7fabbd186700 1 -- 172.21.15.97:0/3527483542 learned_addr learned my addr 172.21.15.97:0/3527483542 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-14T11:56:14.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:14.724+0000 7fabbc985700 1 --2- 172.21.15.97:0/3527483542 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fabb80ffba0 0x7fabb80742b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:14.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:14.724+0000 7fabb7fff700 1 --2- 172.21.15.97:0/3527483542 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabb8100e40 0x7fabb8072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:14.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:14.724+0000 7fabbd186700 1 -- 172.21.15.97:0/3527483542 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fabb80ffba0 msgr2=0x7fabb80742b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:14.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:14.724+0000 7fabbd186700 1 --2- 172.21.15.97:0/3527483542 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fabb80ffba0 0x7fabb80742b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:14.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:14.724+0000 7fabbd186700 1 -- 172.21.15.97:0/3527483542 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabb8100e40 msgr2=0x7fabb8072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:14.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:14.724+0000 7fabbd186700 1 --2- 172.21.15.97:0/3527483542 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabb8100e40 0x7fabb8072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:14.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:14.724+0000 7fabbd186700 1 -- 172.21.15.97:0/3527483542 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabac0095f0 con 0x7fabb81017f0 2023-07-14T11:56:14.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:14.724+0000 7fabbc985700 1 --2- 172.21.15.97:0/3527483542 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fabb80ffba0 0x7fabb80742b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:56:14.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:14.724+0000 7fabb7fff700 1 --2- 172.21.15.97:0/3527483542 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabb8100e40 0x7fabb8072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:56:14.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:14.724+0000 7fabbd186700 1 --2- 172.21.15.97:0/3527483542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabb81017f0 0x7fabb8072e40 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fabb000b970 tx=0x7fabb000bd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:14.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:14.725+0000 7fabb5ffb700 1 -- 172.21.15.97:0/3527483542 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb000c9b0 con 0x7fabb81017f0 2023-07-14T11:56:14.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:14.725+0000 7fabb5ffb700 1 -- 172.21.15.97:0/3527483542 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fabb0014440 con 0x7fabb81017f0 2023-07-14T11:56:14.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:14.725+0000 7fabbebe9700 1 -- 172.21.15.97:0/3527483542 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabb819fc10 con 0x7fabb81017f0 2023-07-14T11:56:14.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:14.725+0000 7fabb5ffb700 1 -- 172.21.15.97:0/3527483542 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb00075d0 con 0x7fabb81017f0 2023-07-14T11:56:14.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:14.725+0000 7fabbebe9700 1 -- 172.21.15.97:0/3527483542 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabb81a0110 con 0x7fabb81017f0 2023-07-14T11:56:14.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:14.726+0000 7fabbebe9700 1 -- 172.21.15.97:0/3527483542 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabb810b3f0 con 0x7fabb81017f0 2023-07-14T11:56:14.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:14.727+0000 7fabb5ffb700 1 -- 172.21.15.97:0/3527483542 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fabb00145b0 con 0x7fabb81017f0 2023-07-14T11:56:14.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:14.727+0000 7fabb5ffb700 1 --2- 172.21.15.97:0/3527483542 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7faba80a11d0 0x7faba80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:14.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:14.727+0000 7fabb5ffb700 1 -- 172.21.15.97:0/3527483542 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4101+0+0 (secure 0 0 0) 0x7fabb0006e40 con 0x7fabb81017f0 2023-07-14T11:56:14.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:14.728+0000 7fabbc985700 1 --2- 172.21.15.97:0/3527483542 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7faba80a11d0 0x7faba80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:14.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:14.728+0000 7fabbc985700 1 --2- 172.21.15.97:0/3527483542 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7faba80a11d0 0x7faba80a3680 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fabac006010 tx=0x7fabac00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:14.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:14.732+0000 7fabb5ffb700 1 -- 172.21.15.97:0/3527483542 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fabb004e790 con 0x7fabb81017f0 2023-07-14T11:56:14.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:14 smithi097 ceph-mon[129313]: purged_snaps scrub starts 2023-07-14T11:56:14.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:14 smithi097 ceph-mon[129313]: purged_snaps scrub ok 2023-07-14T11:56:14.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:14 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T11:56:14.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:14 smithi097 ceph-mon[129313]: osd.4 [v2:172.21.15.93:6808/3151801173,v1:172.21.15.93:6809/3151801173] boot 2023-07-14T11:56:14.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:14 smithi097 ceph-mon[129313]: osdmap e34: 5 total, 5 up, 5 in 2023-07-14T11:56:14.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:14 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T11:56:14.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:14 smithi097 ceph-mon[129313]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T11:56:14.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:14 smithi017 ceph-mon[126339]: purged_snaps scrub starts 2023-07-14T11:56:14.896 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:14 smithi017 ceph-mon[126339]: purged_snaps scrub ok 2023-07-14T11:56:14.896 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:14 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T11:56:14.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:14 smithi017 ceph-mon[126339]: osd.4 [v2:172.21.15.93:6808/3151801173,v1:172.21.15.93:6809/3151801173] boot 2023-07-14T11:56:14.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:14 smithi017 ceph-mon[126339]: osdmap e34: 5 total, 5 up, 5 in 2023-07-14T11:56:14.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:14 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T11:56:14.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:14 smithi017 ceph-mon[126339]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T11:56:14.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:14.925+0000 7fabbebe9700 1 -- 172.21.15.97:0/3527483542 --> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fabb8060950 con 0x7faba80a11d0 2023-07-14T11:56:15.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:14 smithi093 ceph-mon[128922]: purged_snaps scrub starts 2023-07-14T11:56:15.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:14 smithi093 ceph-mon[128922]: purged_snaps scrub ok 2023-07-14T11:56:15.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:14 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T11:56:15.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:14 smithi093 ceph-mon[128922]: osd.4 [v2:172.21.15.93:6808/3151801173,v1:172.21.15.93:6809/3151801173] boot 2023-07-14T11:56:15.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:14 smithi093 ceph-mon[128922]: osdmap e34: 5 total, 5 up, 5 in 2023-07-14T11:56:15.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:14 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T11:56:15.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:14 smithi093 ceph-mon[128922]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T11:56:16.003 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:15 smithi097 ceph-mon[129313]: osdmap e35: 5 total, 5 up, 5 in 2023-07-14T11:56:16.004 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:15 smithi097 ceph-mon[129313]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:56:16.004 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:15 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:56:16.004 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:15 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:56:16.004 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:15 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:56:16.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:15 smithi093 ceph-mon[128922]: osdmap e35: 5 total, 5 up, 5 in 2023-07-14T11:56:16.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:15 smithi093 ceph-mon[128922]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:56:16.115 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:15 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:56:16.115 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:15 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:56:16.115 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:15 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:56:16.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:15 smithi017 ceph-mon[126339]: osdmap e35: 5 total, 5 up, 5 in 2023-07-14T11:56:16.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:15 smithi017 ceph-mon[126339]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:56:16.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:15 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:56:16.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:15 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:56:16.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:15 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:56:16.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:16 smithi097 ceph-mon[129313]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:56:17.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:16 smithi093 ceph-mon[128922]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:56:17.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:16 smithi017 ceph-mon[126339]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:56:18.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:17 smithi093 ceph-mon[128922]: from='client.? 172.21.15.97:0/476775674' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d2af354-0535-4ef9-b166-3ef219498db8"}]: dispatch 2023-07-14T11:56:18.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:17 smithi093 ceph-mon[128922]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d2af354-0535-4ef9-b166-3ef219498db8"}]: dispatch 2023-07-14T11:56:18.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:17 smithi093 ceph-mon[128922]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d2af354-0535-4ef9-b166-3ef219498db8"}]': finished 2023-07-14T11:56:18.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:17 smithi093 ceph-mon[128922]: osdmap e36: 6 total, 5 up, 6 in 2023-07-14T11:56:18.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:17 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T11:56:18.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:17 smithi097 ceph-mon[129313]: from='client.? 172.21.15.97:0/476775674' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d2af354-0535-4ef9-b166-3ef219498db8"}]: dispatch 2023-07-14T11:56:18.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:17 smithi097 ceph-mon[129313]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d2af354-0535-4ef9-b166-3ef219498db8"}]: dispatch 2023-07-14T11:56:18.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:17 smithi097 ceph-mon[129313]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d2af354-0535-4ef9-b166-3ef219498db8"}]': finished 2023-07-14T11:56:18.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:17 smithi097 ceph-mon[129313]: osdmap e36: 6 total, 5 up, 6 in 2023-07-14T11:56:18.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:17 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T11:56:18.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:17 smithi017 ceph-mon[126339]: from='client.? 172.21.15.97:0/476775674' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d2af354-0535-4ef9-b166-3ef219498db8"}]: dispatch 2023-07-14T11:56:18.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:17 smithi017 ceph-mon[126339]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d2af354-0535-4ef9-b166-3ef219498db8"}]: dispatch 2023-07-14T11:56:18.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:17 smithi017 ceph-mon[126339]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d2af354-0535-4ef9-b166-3ef219498db8"}]': finished 2023-07-14T11:56:18.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:17 smithi017 ceph-mon[126339]: osdmap e36: 6 total, 5 up, 6 in 2023-07-14T11:56:18.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:17 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T11:56:19.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:18 smithi093 ceph-mon[128922]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:56:19.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:18 smithi093 ceph-mon[128922]: from='client.? 172.21.15.97:0/706642057' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:56:19.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:18 smithi097 ceph-mon[129313]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:56:19.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:18 smithi097 ceph-mon[129313]: from='client.? 172.21.15.97:0/706642057' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:56:19.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:18 smithi017 ceph-mon[126339]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:56:19.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:18 smithi017 ceph-mon[126339]: from='client.? 172.21.15.97:0/706642057' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:56:20.924 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:20 smithi097 ceph-mon[129313]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:56:21.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:20 smithi093 ceph-mon[128922]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:56:21.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:20 smithi017 ceph-mon[126339]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:56:22.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:22 smithi097 ceph-mon[129313]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:56:23.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:22 smithi093 ceph-mon[128922]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:56:23.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:22 smithi017 ceph-mon[126339]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:56:24.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:24 smithi097 ceph-mon[129313]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:56:24.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:24 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-07-14T11:56:24.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:24 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:56:24.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:24 smithi097 ceph-mon[129313]: Deploying daemon osd.5 on smithi097 2023-07-14T11:56:24.645 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:24 smithi017 ceph-mon[126339]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:56:24.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:24 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-07-14T11:56:24.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:24 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:56:24.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:24 smithi017 ceph-mon[126339]: Deploying daemon osd.5 on smithi097 2023-07-14T11:56:24.862 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:24 smithi093 ceph-mon[128922]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:56:24.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:24 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-07-14T11:56:24.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:24 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:56:24.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:24 smithi093 ceph-mon[128922]: Deploying daemon osd.5 on smithi097 2023-07-14T11:56:27.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:26 smithi093 ceph-mon[128922]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:56:27.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:26 smithi097 ceph-mon[129313]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:56:27.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:26 smithi017 ceph-mon[126339]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:56:29.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:28 smithi093 ceph-mon[128922]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:56:29.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:28 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:29.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:28 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:29.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:28 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:56:29.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:28 smithi097 ceph-mon[129313]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:56:29.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:28 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:29.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:28 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:29.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:28 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:56:29.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:28 smithi017 ceph-mon[126339]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:56:29.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:28 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:29.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:28 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:29.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:28 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:56:31.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:30 smithi097 ceph-mon[129313]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:56:31.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:30 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:31.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:30 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:31.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:30 smithi093 ceph-mon[128922]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:56:31.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:30 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:31.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:30 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:31.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:30 smithi017 ceph-mon[126339]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:56:31.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:30 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:31.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:30 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:31.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:31.188+0000 7fabb5ffb700 1 -- 172.21.15.97:0/3527483542 <== mgr.14152 v2:172.21.15.17:6800/4237449858 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fabb8060950 con 0x7faba80a11d0 2023-07-14T11:56:31.171 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 5 on host 'smithi097' 2023-07-14T11:56:31.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:31.190+0000 7fabbebe9700 1 -- 172.21.15.97:0/3527483542 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7faba80a11d0 msgr2=0x7faba80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:31.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:31.190+0000 7fabbebe9700 1 --2- 172.21.15.97:0/3527483542 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7faba80a11d0 0x7faba80a3680 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fabac006010 tx=0x7fabac00b540 comp rx=0 tx=0).stop 2023-07-14T11:56:31.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:31.190+0000 7fabbebe9700 1 -- 172.21.15.97:0/3527483542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabb81017f0 msgr2=0x7fabb8072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:31.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:31.190+0000 7fabbebe9700 1 --2- 172.21.15.97:0/3527483542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabb81017f0 0x7fabb8072e40 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fabb000b970 tx=0x7fabb000bd30 comp rx=0 tx=0).stop 2023-07-14T11:56:31.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:31.191+0000 7fabbebe9700 1 -- 172.21.15.97:0/3527483542 shutdown_connections 2023-07-14T11:56:31.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:31.191+0000 7fabbebe9700 1 --2- 172.21.15.97:0/3527483542 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7faba80a11d0 0x7faba80a3680 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:31.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:31.191+0000 7fabbebe9700 1 --2- 172.21.15.97:0/3527483542 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fabb80ffba0 0x7fabb80742b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:31.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:31.191+0000 7fabbebe9700 1 --2- 172.21.15.97:0/3527483542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabb81017f0 0x7fabb8072e40 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:31.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:31.191+0000 7fabbebe9700 1 --2- 172.21.15.97:0/3527483542 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabb8100e40 0x7fabb8072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:31.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:31.191+0000 7fabbebe9700 1 -- 172.21.15.97:0/3527483542 >> 172.21.15.97:0/3527483542 conn(0x7fabb80fb1f0 msgr2=0x7fabb80fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:31.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:31.191+0000 7fabbebe9700 1 -- 172.21.15.97:0/3527483542 shutdown_connections 2023-07-14T11:56:31.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:56:31.191+0000 7fabbebe9700 1 -- 172.21.15.97:0/3527483542 wait complete. 2023-07-14T11:56:31.679 DEBUG:teuthology.orchestra.run.smithi097:osd.5> sudo journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@osd.5.service 2023-07-14T11:56:31.682 INFO:tasks.cephadm:Deploying osd.6 on smithi017 with /dev/vg_nvme/lv_2... 2023-07-14T11:56:31.682 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-07-14T11:56:31.710 INFO:journalctl@ceph.osd.5.smithi097.stdout:-- Logs begin at Fri 2023-07-14 11:24:16 UTC. -- 2023-07-14T11:56:32.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:31 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:56:32.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:31 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:56:32.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:31 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:32.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:31 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:32.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:31 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:32.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:31 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:56:32.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:31 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:56:32.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:31 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:56:32.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:31 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:32.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:31 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:32.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:31 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:32.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:31 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:56:32.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:31 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:56:32.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:31 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:56:32.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:31 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:32.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:31 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:32.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:31 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:32.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:31 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:56:32.442 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T11:56:32.492 INFO:journalctl@ceph.osd.5.smithi097.stdout:Jul 14 11:56:32 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-5[140631]: 2023-07-14T11:56:32.228+0000 7f925b8f0640 -1 osd.5 0 log_to_monitors true 2023-07-14T11:56:32.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:32 smithi097 ceph-mon[129313]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:56:32.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:32 smithi097 ceph-mon[129313]: from='osd.5 [v2:172.21.15.97:6808/2184413924,v1:172.21.15.97:6809/2184413924]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-14T11:56:32.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:32 smithi097 ceph-mon[129313]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-14T11:56:33.011 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:32 smithi017 ceph-mon[126339]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:56:33.011 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:32 smithi017 ceph-mon[126339]: from='osd.5 [v2:172.21.15.97:6808/2184413924,v1:172.21.15.97:6809/2184413924]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-14T11:56:33.011 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:32 smithi017 ceph-mon[126339]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-14T11:56:33.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:32 smithi093 ceph-mon[128922]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:56:33.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:32 smithi093 ceph-mon[128922]: from='osd.5 [v2:172.21.15.97:6808/2184413924,v1:172.21.15.97:6809/2184413924]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-14T11:56:33.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:32 smithi093 ceph-mon[128922]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-14T11:56:34.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:34 smithi017 ceph-mon[126339]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-07-14T11:56:34.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:34 smithi017 ceph-mon[126339]: osdmap e37: 6 total, 5 up, 6 in 2023-07-14T11:56:34.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:34 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T11:56:34.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:34 smithi017 ceph-mon[126339]: from='osd.5 [v2:172.21.15.97:6808/2184413924,v1:172.21.15.97:6809/2184413924]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-07-14T11:56:34.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:34 smithi017 ceph-mon[126339]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-07-14T11:56:34.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:34 smithi017 ceph-mon[126339]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:56:34.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:34 smithi017 ceph-mon[126339]: Detected new or changed devices on smithi097 2023-07-14T11:56:34.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:34 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:34.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:34 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:34.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:34 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:56:34.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:34 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:56:34.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:34 smithi017 ceph-mon[126339]: Adjusting osd_memory_target on smithi097 to 8584M 2023-07-14T11:56:34.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:34 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:34.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:34 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:56:34.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:34 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:56:34.398 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:34 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:34.493 INFO:teuthology.orchestra.run.smithi017.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-07-14T11:56:34.493 INFO:teuthology.orchestra.run.smithi017.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-07-14T11:56:34.494 INFO:teuthology.orchestra.run.smithi017.stderr: stderr: 10+0 records in 2023-07-14T11:56:34.494 INFO:teuthology.orchestra.run.smithi017.stderr:10+0 records out 2023-07-14T11:56:34.494 INFO:teuthology.orchestra.run.smithi017.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0299809 s, 350 MB/s 2023-07-14T11:56:34.494 INFO:teuthology.orchestra.run.smithi017.stderr:--> Zapping successful for: 2023-07-14T11:56:34.612 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:34 smithi093 ceph-mon[128922]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-07-14T11:56:34.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:34 smithi093 ceph-mon[128922]: osdmap e37: 6 total, 5 up, 6 in 2023-07-14T11:56:34.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:34 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T11:56:34.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:34 smithi093 ceph-mon[128922]: from='osd.5 [v2:172.21.15.97:6808/2184413924,v1:172.21.15.97:6809/2184413924]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-07-14T11:56:34.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:34 smithi093 ceph-mon[128922]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-07-14T11:56:34.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:34 smithi093 ceph-mon[128922]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:56:34.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:34 smithi093 ceph-mon[128922]: Detected new or changed devices on smithi097 2023-07-14T11:56:34.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:34 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:34.614 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:34 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:34.614 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:34 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:56:34.614 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:34 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:56:34.614 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:34 smithi093 ceph-mon[128922]: Adjusting osd_memory_target on smithi097 to 8584M 2023-07-14T11:56:34.614 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:34 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:34.614 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:34 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:56:34.614 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:34 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:56:34.615 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:34 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:34.636 INFO:journalctl@ceph.osd.5.smithi097.stdout:Jul 14 11:56:34 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-5[140631]: 2023-07-14T11:56:34.166+0000 7f9250f68700 -1 osd.5 0 waiting for initial osdmap 2023-07-14T11:56:34.636 INFO:journalctl@ceph.osd.5.smithi097.stdout:Jul 14 11:56:34 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-5[140631]: 2023-07-14T11:56:34.173+0000 7f924cd5d700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T11:56:34.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:34 smithi097 ceph-mon[129313]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-07-14T11:56:34.637 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:34 smithi097 ceph-mon[129313]: osdmap e37: 6 total, 5 up, 6 in 2023-07-14T11:56:34.637 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:34 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T11:56:34.637 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:34 smithi097 ceph-mon[129313]: from='osd.5 [v2:172.21.15.97:6808/2184413924,v1:172.21.15.97:6809/2184413924]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-07-14T11:56:34.637 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:34 smithi097 ceph-mon[129313]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-07-14T11:56:34.638 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:34 smithi097 ceph-mon[129313]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T11:56:34.638 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:34 smithi097 ceph-mon[129313]: Detected new or changed devices on smithi097 2023-07-14T11:56:34.638 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:34 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:34.639 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:34 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:34.639 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:34 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:56:34.639 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:34 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:56:34.639 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:34 smithi097 ceph-mon[129313]: Adjusting osd_memory_target on smithi097 to 8584M 2023-07-14T11:56:34.640 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:34 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:34.640 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:34 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:56:34.640 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:34 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:56:34.640 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:34 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:35.029 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi017:vg_nvme/lv_2 2023-07-14T11:56:35.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:35 smithi017 ceph-mon[126339]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2023-07-14T11:56:35.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:35 smithi017 ceph-mon[126339]: osdmap e38: 6 total, 5 up, 6 in 2023-07-14T11:56:35.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:35 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T11:56:35.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:35 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T11:56:35.612 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:35 smithi093 ceph-mon[128922]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2023-07-14T11:56:35.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:35 smithi093 ceph-mon[128922]: osdmap e38: 6 total, 5 up, 6 in 2023-07-14T11:56:35.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:35 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T11:56:35.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:35 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T11:56:35.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:35 smithi097 ceph-mon[129313]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2023-07-14T11:56:35.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:35 smithi097 ceph-mon[129313]: osdmap e38: 6 total, 5 up, 6 in 2023-07-14T11:56:35.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:35 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T11:56:35.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:35 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T11:56:35.765 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T11:56:36.336 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:36 smithi017 ceph-mon[126339]: purged_snaps scrub starts 2023-07-14T11:56:36.337 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:36 smithi017 ceph-mon[126339]: purged_snaps scrub ok 2023-07-14T11:56:36.337 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:36 smithi017 ceph-mon[126339]: osd.5 [v2:172.21.15.97:6808/2184413924,v1:172.21.15.97:6809/2184413924] boot 2023-07-14T11:56:36.337 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:36 smithi017 ceph-mon[126339]: osdmap e39: 6 total, 6 up, 6 in 2023-07-14T11:56:36.338 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:36 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T11:56:36.338 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:36 smithi017 ceph-mon[126339]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:56:36.338 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:36 smithi017 ceph-mon[126339]: osdmap e40: 6 total, 6 up, 6 in 2023-07-14T11:56:36.612 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:36 smithi093 ceph-mon[128922]: purged_snaps scrub starts 2023-07-14T11:56:36.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:36 smithi093 ceph-mon[128922]: purged_snaps scrub ok 2023-07-14T11:56:36.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:36 smithi093 ceph-mon[128922]: osd.5 [v2:172.21.15.97:6808/2184413924,v1:172.21.15.97:6809/2184413924] boot 2023-07-14T11:56:36.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:36 smithi093 ceph-mon[128922]: osdmap e39: 6 total, 6 up, 6 in 2023-07-14T11:56:36.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:36 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T11:56:36.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:36 smithi093 ceph-mon[128922]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:56:36.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:36 smithi093 ceph-mon[128922]: osdmap e40: 6 total, 6 up, 6 in 2023-07-14T11:56:36.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:36 smithi097 ceph-mon[129313]: purged_snaps scrub starts 2023-07-14T11:56:36.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:36 smithi097 ceph-mon[129313]: purged_snaps scrub ok 2023-07-14T11:56:36.637 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:36 smithi097 ceph-mon[129313]: osd.5 [v2:172.21.15.97:6808/2184413924,v1:172.21.15.97:6809/2184413924] boot 2023-07-14T11:56:36.637 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:36 smithi097 ceph-mon[129313]: osdmap e39: 6 total, 6 up, 6 in 2023-07-14T11:56:36.637 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:36 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T11:56:36.637 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:36 smithi097 ceph-mon[129313]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:56:36.638 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:36 smithi097 ceph-mon[129313]: osdmap e40: 6 total, 6 up, 6 in 2023-07-14T11:56:37.351 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:37.368+0000 7f548644b700 1 -- 172.21.15.17:0/3950071706 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54800ffb90 msgr2=0x7f54800fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:37.352 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:37.368+0000 7f548644b700 1 --2- 172.21.15.17:0/3950071706 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54800ffb90 0x7f54800fffa0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5470009990 tx=0x7f5470009ca0 comp rx=0 tx=0).stop 2023-07-14T11:56:37.352 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:37.368+0000 7f548644b700 1 -- 172.21.15.17:0/3950071706 shutdown_connections 2023-07-14T11:56:37.352 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:37.368+0000 7f548644b700 1 --2- 172.21.15.17:0/3950071706 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5480101720 0x7f5480107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:37.352 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:37.368+0000 7f548644b700 1 --2- 172.21.15.17:0/3950071706 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54800ffb90 0x7f54800fffa0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:37.352 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:37.368+0000 7f548644b700 1 --2- 172.21.15.17:0/3950071706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5480100d90 0x7f54801011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:37.353 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:37.368+0000 7f548644b700 1 -- 172.21.15.17:0/3950071706 >> 172.21.15.17:0/3950071706 conn(0x7f54800fb1a0 msgr2=0x7f54800fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:37.353 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:37.369+0000 7f548644b700 1 -- 172.21.15.17:0/3950071706 shutdown_connections 2023-07-14T11:56:37.353 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:37.369+0000 7f548644b700 1 -- 172.21.15.17:0/3950071706 wait complete. 2023-07-14T11:56:37.353 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:37.370+0000 7f548644b700 1 Processor -- start 2023-07-14T11:56:37.355 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:37.370+0000 7f548644b700 1 -- start start 2023-07-14T11:56:37.355 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:37.370+0000 7f548644b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54800ffb90 0x7f5480197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:37.355 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:37.370+0000 7f548644b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5480100d90 0x7f5480198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:37.355 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:37.370+0000 7f548644b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5480101720 0x7f548019f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:37.355 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:37.370+0000 7f548644b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548019f660 con 0x7f5480101720 2023-07-14T11:56:37.355 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:37.370+0000 7f548644b700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548019f7a0 con 0x7f5480100d90 2023-07-14T11:56:37.356 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:37.370+0000 7f548644b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548019f930 con 0x7f54800ffb90 2023-07-14T11:56:37.356 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:37.370+0000 7f54849e8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5480101720 0x7f548019f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:37.356 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:37.370+0000 7f547ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54800ffb90 0x7f5480197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:37.356 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:37.370+0000 7f547f7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5480100d90 0x7f5480198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:37.356 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:37.370+0000 7f547ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54800ffb90 0x7f5480197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.17:53216/0 (socket says 172.21.15.17:53216) 2023-07-14T11:56:37.356 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:37.370+0000 7f547ffff700 1 -- 172.21.15.17:0/2181691807 learned_addr learned my addr 172.21.15.17:0/2181691807 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:56:37.356 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:37.371+0000 7f547ffff700 1 -- 172.21.15.17:0/2181691807 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5480100d90 msgr2=0x7f5480198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:37.357 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:37.371+0000 7f547ffff700 1 --2- 172.21.15.17:0/2181691807 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5480100d90 0x7f5480198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:37.357 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:37.371+0000 7f547ffff700 1 -- 172.21.15.17:0/2181691807 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5480101720 msgr2=0x7f548019f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:37.357 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:37.371+0000 7f547ffff700 1 --2- 172.21.15.17:0/2181691807 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5480101720 0x7f548019f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:37.357 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:37.371+0000 7f547ffff700 1 -- 172.21.15.17:0/2181691807 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54700095f0 con 0x7f54800ffb90 2023-07-14T11:56:37.357 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:37.371+0000 7f547f7fe700 1 --2- 172.21.15.17:0/2181691807 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5480100d90 0x7f5480198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:56:37.357 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:37.371+0000 7f547ffff700 1 --2- 172.21.15.17:0/2181691807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54800ffb90 0x7f5480197ad0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5470004740 tx=0x7f5470009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:37.357 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:37.371+0000 7f547d7fa700 1 -- 172.21.15.17:0/2181691807 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547001d070 con 0x7f54800ffb90 2023-07-14T11:56:37.357 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:37.371+0000 7f548644b700 1 -- 172.21.15.17:0/2181691807 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f548019fbb0 con 0x7f54800ffb90 2023-07-14T11:56:37.358 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:37.371+0000 7f547d7fa700 1 -- 172.21.15.17:0/2181691807 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5470021410 con 0x7f54800ffb90 2023-07-14T11:56:37.358 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:37.371+0000 7f547d7fa700 1 -- 172.21.15.17:0/2181691807 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5470017650 con 0x7f54800ffb90 2023-07-14T11:56:37.358 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:37.372+0000 7f548644b700 1 -- 172.21.15.17:0/2181691807 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f548019ffc0 con 0x7f54800ffb90 2023-07-14T11:56:37.358 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:37.372+0000 7f548644b700 1 -- 172.21.15.17:0/2181691807 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5480192210 con 0x7f54800ffb90 2023-07-14T11:56:37.359 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:37.375+0000 7f547d7fa700 1 -- 172.21.15.17:0/2181691807 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5470021580 con 0x7f54800ffb90 2023-07-14T11:56:37.359 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:37.376+0000 7f547d7fa700 1 --2- 172.21.15.17:0/2181691807 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f54680a1150 0x7f54680a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:56:37.359 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:37.376+0000 7f547d7fa700 1 -- 172.21.15.17:0/2181691807 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4562+0+0 (secure 0 0 0) 0x7f54700cd1d0 con 0x7f54800ffb90 2023-07-14T11:56:37.359 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:37.376+0000 7f547f7fe700 1 --2- 172.21.15.17:0/2181691807 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f54680a1150 0x7f54680a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:56:37.359 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:37.377+0000 7f547f7fe700 1 --2- 172.21.15.17:0/2181691807 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f54680a1150 0x7f54680a3600 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5474011490 tx=0x7f5474009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:56:37.360 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:37.378+0000 7f547d7fa700 1 -- 172.21.15.17:0/2181691807 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f54700977b0 con 0x7f54800ffb90 2023-07-14T11:56:37.577 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:37.595+0000 7f548644b700 1 -- 172.21.15.17:0/2181691807 --> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f5480060950 con 0x7f54680a1150 2023-07-14T11:56:38.612 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:38 smithi093 ceph-mon[128922]: osdmap e41: 6 total, 6 up, 6 in 2023-07-14T11:56:38.612 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:38 smithi093 ceph-mon[128922]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:56:38.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:38 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:56:38.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:38 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:56:38.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:38 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:56:38.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:38 smithi093 ceph-mon[128922]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:56:38.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:38 smithi097 ceph-mon[129313]: osdmap e41: 6 total, 6 up, 6 in 2023-07-14T11:56:38.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:38 smithi097 ceph-mon[129313]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:56:38.637 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:38 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:56:38.637 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:38 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:56:38.638 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:38 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:56:38.638 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:38 smithi097 ceph-mon[129313]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:56:38.645 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:38 smithi017 ceph-mon[126339]: osdmap e41: 6 total, 6 up, 6 in 2023-07-14T11:56:38.645 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:38 smithi017 ceph-mon[126339]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:56:38.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:38 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:56:38.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:38 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:56:38.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:38 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:56:38.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:38 smithi017 ceph-mon[126339]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:56:39.360 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:39 smithi017 ceph-mon[126339]: osdmap e42: 6 total, 6 up, 6 in 2023-07-14T11:56:39.612 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:39 smithi093 ceph-mon[128922]: osdmap e42: 6 total, 6 up, 6 in 2023-07-14T11:56:39.635 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:39 smithi097 ceph-mon[129313]: osdmap e42: 6 total, 6 up, 6 in 2023-07-14T11:56:40.612 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:40 smithi093 ceph-mon[128922]: osdmap e43: 6 total, 6 up, 6 in 2023-07-14T11:56:40.612 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:40 smithi093 ceph-mon[128922]: pgmap v140: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:56:40.635 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:40 smithi097 ceph-mon[129313]: osdmap e43: 6 total, 6 up, 6 in 2023-07-14T11:56:40.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:40 smithi097 ceph-mon[129313]: pgmap v140: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:56:40.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:40 smithi017 ceph-mon[126339]: osdmap e43: 6 total, 6 up, 6 in 2023-07-14T11:56:40.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:40 smithi017 ceph-mon[126339]: pgmap v140: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:56:41.612 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:41 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/2522321202' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9cdf0c4b-62dc-4168-864e-a9d1650a9ea8"}]: dispatch 2023-07-14T11:56:41.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:41 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/2522321202' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9cdf0c4b-62dc-4168-864e-a9d1650a9ea8"}]': finished 2023-07-14T11:56:41.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:41 smithi093 ceph-mon[128922]: osdmap e44: 7 total, 6 up, 7 in 2023-07-14T11:56:41.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:41 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T11:56:41.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:41 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/2522321202' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9cdf0c4b-62dc-4168-864e-a9d1650a9ea8"}]: dispatch 2023-07-14T11:56:41.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:41 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/2522321202' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9cdf0c4b-62dc-4168-864e-a9d1650a9ea8"}]': finished 2023-07-14T11:56:41.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:41 smithi097 ceph-mon[129313]: osdmap e44: 7 total, 6 up, 7 in 2023-07-14T11:56:41.637 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:41 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T11:56:41.645 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:41 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/2522321202' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9cdf0c4b-62dc-4168-864e-a9d1650a9ea8"}]: dispatch 2023-07-14T11:56:41.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:41 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/2522321202' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9cdf0c4b-62dc-4168-864e-a9d1650a9ea8"}]': finished 2023-07-14T11:56:41.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:41 smithi017 ceph-mon[126339]: osdmap e44: 7 total, 6 up, 7 in 2023-07-14T11:56:41.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:41 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T11:56:42.612 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:42 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/433462392' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:56:42.612 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:42 smithi093 ceph-mon[128922]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 127 KiB/s, 0 objects/s recovering 2023-07-14T11:56:42.635 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:42 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/433462392' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:56:42.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:42 smithi097 ceph-mon[129313]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 127 KiB/s, 0 objects/s recovering 2023-07-14T11:56:42.645 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:42 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/433462392' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:56:42.645 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:42 smithi017 ceph-mon[126339]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 127 KiB/s, 0 objects/s recovering 2023-07-14T11:56:44.635 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:44 smithi097 ceph-mon[129313]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-14T11:56:44.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:44 smithi017 ceph-mon[126339]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-14T11:56:44.862 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:44 smithi093 ceph-mon[128922]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-14T11:56:46.896 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:46 smithi017 ceph-mon[126339]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2023-07-14T11:56:47.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:46 smithi093 ceph-mon[128922]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2023-07-14T11:56:47.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:46 smithi097 ceph-mon[129313]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2023-07-14T11:56:48.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:48 smithi017 ceph-mon[126339]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-07-14T11:56:48.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:48 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-07-14T11:56:48.896 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:48 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:56:49.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:48 smithi093 ceph-mon[128922]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-07-14T11:56:49.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:48 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-07-14T11:56:49.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:48 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:56:49.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:48 smithi097 ceph-mon[129313]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-07-14T11:56:49.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:48 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-07-14T11:56:49.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:48 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:56:49.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:49 smithi017 ceph-mon[126339]: Deploying daemon osd.6 on smithi017 2023-07-14T11:56:50.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:49 smithi093 ceph-mon[128922]: Deploying daemon osd.6 on smithi017 2023-07-14T11:56:50.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:49 smithi097 ceph-mon[129313]: Deploying daemon osd.6 on smithi017 2023-07-14T11:56:51.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:50 smithi093 ceph-mon[128922]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-14T11:56:51.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:50 smithi097 ceph-mon[129313]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-14T11:56:51.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:50 smithi017 ceph-mon[126339]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-14T11:56:53.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:52 smithi093 ceph-mon[128922]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 53 KiB/s, 0 objects/s recovering 2023-07-14T11:56:53.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:52 smithi097 ceph-mon[129313]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 53 KiB/s, 0 objects/s recovering 2023-07-14T11:56:53.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:52 smithi017 ceph-mon[126339]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 53 KiB/s, 0 objects/s recovering 2023-07-14T11:56:54.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:54 smithi097 ceph-mon[129313]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:56:54.645 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:54 smithi017 ceph-mon[126339]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:56:54.862 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:54 smithi093 ceph-mon[128922]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:56:56.568 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:56 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:56.568 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:56 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:56.568 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:56 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:56:56.569 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:56 smithi017 ceph-mon[126339]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:56:56.612 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:56 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:56.612 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:56 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:56.612 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:56 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:56:56.613 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:56 smithi093 ceph-mon[128922]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:56:56.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:56 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:56.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:56 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:56.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:56 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:56:56.637 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:56 smithi097 ceph-mon[129313]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:56:58.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:58 smithi017 ceph-mon[126339]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:56:58.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:58 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:58.896 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:58 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:58.896 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:58 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:56:58.896 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:58 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:56:58.896 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:56:58 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:59.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:58 smithi093 ceph-mon[128922]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:56:59.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:58 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:59.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:58 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:59.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:58 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:56:59.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:58 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:56:59.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:56:58 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:59.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:58 smithi097 ceph-mon[129313]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:56:59.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:58 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:59.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:58 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:59.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:58 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:56:59.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:58 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:56:59.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:56:58 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:56:59.147 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:59.166+0000 7f547d7fa700 1 -- 172.21.15.17:0/2181691807 <== mgr.14152 v2:172.21.15.17:6800/4237449858 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5480060950 con 0x7f54680a1150 2023-07-14T11:56:59.147 INFO:teuthology.orchestra.run.smithi017.stdout:Created osd(s) 6 on host 'smithi017' 2023-07-14T11:56:59.150 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:59.169+0000 7f548644b700 1 -- 172.21.15.17:0/2181691807 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f54680a1150 msgr2=0x7f54680a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:59.150 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:59.169+0000 7f548644b700 1 --2- 172.21.15.17:0/2181691807 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f54680a1150 0x7f54680a3600 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5474011490 tx=0x7f5474009260 comp rx=0 tx=0).stop 2023-07-14T11:56:59.150 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:59.169+0000 7f548644b700 1 -- 172.21.15.17:0/2181691807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54800ffb90 msgr2=0x7f5480197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:56:59.150 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:59.169+0000 7f548644b700 1 --2- 172.21.15.17:0/2181691807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54800ffb90 0x7f5480197ad0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5470004740 tx=0x7f5470009580 comp rx=0 tx=0).stop 2023-07-14T11:56:59.151 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:59.169+0000 7f548644b700 1 -- 172.21.15.17:0/2181691807 shutdown_connections 2023-07-14T11:56:59.151 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:59.169+0000 7f548644b700 1 --2- 172.21.15.17:0/2181691807 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f54680a1150 0x7f54680a3600 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:59.151 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:59.169+0000 7f548644b700 1 --2- 172.21.15.17:0/2181691807 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5480101720 0x7f548019f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:59.151 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:59.169+0000 7f548644b700 1 --2- 172.21.15.17:0/2181691807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54800ffb90 0x7f5480197ad0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:59.151 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:59.169+0000 7f548644b700 1 --2- 172.21.15.17:0/2181691807 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5480100d90 0x7f5480198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:56:59.151 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:59.169+0000 7f548644b700 1 -- 172.21.15.17:0/2181691807 >> 172.21.15.17:0/2181691807 conn(0x7f54800fb1a0 msgr2=0x7f54801066f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:56:59.152 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:59.169+0000 7f548644b700 1 -- 172.21.15.17:0/2181691807 shutdown_connections 2023-07-14T11:56:59.152 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:56:59.169+0000 7f548644b700 1 -- 172.21.15.17:0/2181691807 wait complete. 2023-07-14T11:56:59.630 DEBUG:teuthology.orchestra.run.smithi017:osd.6> sudo journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@osd.6.service 2023-07-14T11:56:59.636 INFO:tasks.cephadm:Deploying osd.7 on smithi093 with /dev/vg_nvme/lv_2... 2023-07-14T11:56:59.637 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-07-14T11:56:59.716 INFO:journalctl@ceph.osd.6.smithi017.stdout:-- Logs begin at Fri 2023-07-14 11:32:03 UTC. -- 2023-07-14T11:57:00.211 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.b/config 2023-07-14T11:57:00.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:00 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:00.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:00 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:00.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:00 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:57:00.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:00 smithi097 ceph-mon[129313]: from='osd.6 [v2:172.21.15.17:6818/50486581,v1:172.21.15.17:6819/50486581]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-14T11:57:00.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:00 smithi097 ceph-mon[129313]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-14T11:57:00.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:00 smithi097 ceph-mon[129313]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:57:00.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:00 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:00.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:00 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:00.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:00 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:57:00.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:00 smithi017 ceph-mon[126339]: from='osd.6 [v2:172.21.15.17:6818/50486581,v1:172.21.15.17:6819/50486581]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-14T11:57:00.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:00 smithi017 ceph-mon[126339]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-14T11:57:00.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:00 smithi017 ceph-mon[126339]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:57:00.397 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:00 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:00.397 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:00 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:00.398 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:00 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:57:00.398 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:00 smithi093 ceph-mon[128922]: from='osd.6 [v2:172.21.15.17:6818/50486581,v1:172.21.15.17:6819/50486581]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-14T11:57:00.398 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:00 smithi093 ceph-mon[128922]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-14T11:57:00.398 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:00 smithi093 ceph-mon[128922]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:57:01.209 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:01 smithi093 ceph-mon[128922]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-07-14T11:57:01.209 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:01 smithi093 ceph-mon[128922]: osdmap e45: 7 total, 6 up, 7 in 2023-07-14T11:57:01.209 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:01 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T11:57:01.210 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:01 smithi093 ceph-mon[128922]: from='osd.6 [v2:172.21.15.17:6818/50486581,v1:172.21.15.17:6819/50486581]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2023-07-14T11:57:01.210 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:01 smithi093 ceph-mon[128922]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2023-07-14T11:57:01.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:01 smithi017 ceph-mon[126339]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-07-14T11:57:01.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:01 smithi017 ceph-mon[126339]: osdmap e45: 7 total, 6 up, 7 in 2023-07-14T11:57:01.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:01 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T11:57:01.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:01 smithi017 ceph-mon[126339]: from='osd.6 [v2:172.21.15.17:6818/50486581,v1:172.21.15.17:6819/50486581]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2023-07-14T11:57:01.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:01 smithi017 ceph-mon[126339]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2023-07-14T11:57:01.396 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 11:57:01 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T11:57:01.171+0000 7f7234266700 -1 osd.6 0 waiting for initial osdmap 2023-07-14T11:57:01.397 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 11:57:01 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T11:57:01.182+0000 7f722e858700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T11:57:01.635 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:01 smithi097 ceph-mon[129313]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-07-14T11:57:01.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:01 smithi097 ceph-mon[129313]: osdmap e45: 7 total, 6 up, 7 in 2023-07-14T11:57:01.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:01 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T11:57:01.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:01 smithi097 ceph-mon[129313]: from='osd.6 [v2:172.21.15.17:6818/50486581,v1:172.21.15.17:6819/50486581]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2023-07-14T11:57:01.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:01 smithi097 ceph-mon[129313]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2023-07-14T11:57:01.676 INFO:teuthology.orchestra.run.smithi093.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-07-14T11:57:01.676 INFO:teuthology.orchestra.run.smithi093.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-07-14T11:57:01.676 INFO:teuthology.orchestra.run.smithi093.stderr: stderr: 10+0 records in 2023-07-14T11:57:01.676 INFO:teuthology.orchestra.run.smithi093.stderr:10+0 records out 2023-07-14T11:57:01.676 INFO:teuthology.orchestra.run.smithi093.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113065 s, 927 MB/s 2023-07-14T11:57:01.677 INFO:teuthology.orchestra.run.smithi093.stderr:--> Zapping successful for: 2023-07-14T11:57:02.058 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi093:vg_nvme/lv_2 2023-07-14T11:57:02.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:02 smithi093 ceph-mon[128922]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2023-07-14T11:57:02.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:02 smithi093 ceph-mon[128922]: osdmap e46: 7 total, 6 up, 7 in 2023-07-14T11:57:02.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:02 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T11:57:02.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:02 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T11:57:02.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:02 smithi093 ceph-mon[128922]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:57:02.608 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.b/config 2023-07-14T11:57:02.617 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:02 smithi017 ceph-mon[126339]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2023-07-14T11:57:02.618 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:02 smithi017 ceph-mon[126339]: osdmap e46: 7 total, 6 up, 7 in 2023-07-14T11:57:02.618 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:02 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T11:57:02.618 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:02 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T11:57:02.618 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:02 smithi017 ceph-mon[126339]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:57:02.635 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:02 smithi097 ceph-mon[129313]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2023-07-14T11:57:02.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:02 smithi097 ceph-mon[129313]: osdmap e46: 7 total, 6 up, 7 in 2023-07-14T11:57:02.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:02 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T11:57:02.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:02 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T11:57:02.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:02 smithi097 ceph-mon[129313]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T11:57:03.605 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:03 smithi093 ceph-mon[128922]: purged_snaps scrub starts 2023-07-14T11:57:03.606 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:03 smithi093 ceph-mon[128922]: purged_snaps scrub ok 2023-07-14T11:57:03.606 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:03 smithi093 ceph-mon[128922]: osd.6 [v2:172.21.15.17:6818/50486581,v1:172.21.15.17:6819/50486581] boot 2023-07-14T11:57:03.606 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:03 smithi093 ceph-mon[128922]: osdmap e47: 7 total, 7 up, 7 in 2023-07-14T11:57:03.606 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:03 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T11:57:03.606 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:03 smithi093 ceph-mon[128922]: Detected new or changed devices on smithi017 2023-07-14T11:57:03.606 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:03 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:03.607 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:03 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:03.607 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:03 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:57:03.607 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:03 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:57:03.607 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:03 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:57:03.607 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:03 smithi093 ceph-mon[128922]: Adjusting osd_memory_target on smithi017 to 5757M 2023-07-14T11:57:03.607 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:03 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:03.607 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:03 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:57:03.607 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:03 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:57:03.608 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:03 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:03.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:03 smithi097 ceph-mon[129313]: purged_snaps scrub starts 2023-07-14T11:57:03.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:03 smithi097 ceph-mon[129313]: purged_snaps scrub ok 2023-07-14T11:57:03.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:03 smithi097 ceph-mon[129313]: osd.6 [v2:172.21.15.17:6818/50486581,v1:172.21.15.17:6819/50486581] boot 2023-07-14T11:57:03.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:03 smithi097 ceph-mon[129313]: osdmap e47: 7 total, 7 up, 7 in 2023-07-14T11:57:03.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:03 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T11:57:03.637 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:03 smithi097 ceph-mon[129313]: Detected new or changed devices on smithi017 2023-07-14T11:57:03.637 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:03 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:03.637 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:03 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:03.637 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:03 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:57:03.637 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:03 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:57:03.638 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:03 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:57:03.638 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:03 smithi097 ceph-mon[129313]: Adjusting osd_memory_target on smithi017 to 5757M 2023-07-14T11:57:03.638 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:03 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:03.638 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:03 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:57:03.639 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:03 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:57:03.639 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:03 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:03.645 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:03 smithi017 ceph-mon[126339]: purged_snaps scrub starts 2023-07-14T11:57:03.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:03 smithi017 ceph-mon[126339]: purged_snaps scrub ok 2023-07-14T11:57:03.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:03 smithi017 ceph-mon[126339]: osd.6 [v2:172.21.15.17:6818/50486581,v1:172.21.15.17:6819/50486581] boot 2023-07-14T11:57:03.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:03 smithi017 ceph-mon[126339]: osdmap e47: 7 total, 7 up, 7 in 2023-07-14T11:57:03.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:03 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T11:57:03.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:03 smithi017 ceph-mon[126339]: Detected new or changed devices on smithi017 2023-07-14T11:57:03.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:03 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:03.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:03 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:03.647 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:03 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:57:03.647 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:03 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:57:03.647 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:03 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:57:03.647 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:03 smithi017 ceph-mon[126339]: Adjusting osd_memory_target on smithi017 to 5757M 2023-07-14T11:57:03.647 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:03 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:03.647 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:03 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:57:03.647 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:03 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:57:03.647 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:03 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:03.808 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:03.824+0000 7ff09fb57700 1 -- 172.21.15.93:0/4118854522 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff098100d90 msgr2=0x7ff0981011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:03.808 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:03.824+0000 7ff09fb57700 1 --2- 172.21.15.93:0/4118854522 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff098100d90 0x7ff0981011e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff094009990 tx=0x7ff094009ca0 comp rx=0 tx=0).stop 2023-07-14T11:57:03.808 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:03.824+0000 7ff09fb57700 1 -- 172.21.15.93:0/4118854522 shutdown_connections 2023-07-14T11:57:03.808 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:03.824+0000 7ff09fb57700 1 --2- 172.21.15.93:0/4118854522 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff098101720 0x7ff098107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:03.808 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:03.824+0000 7ff09fb57700 1 --2- 172.21.15.93:0/4118854522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0980ffb90 0x7ff0980fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:03.809 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:03.824+0000 7ff09fb57700 1 --2- 172.21.15.93:0/4118854522 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff098100d90 0x7ff0981011e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:03.809 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:03.824+0000 7ff09fb57700 1 -- 172.21.15.93:0/4118854522 >> 172.21.15.93:0/4118854522 conn(0x7ff0980fb1a0 msgr2=0x7ff0980fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:03.809 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:03.824+0000 7ff09fb57700 1 -- 172.21.15.93:0/4118854522 shutdown_connections 2023-07-14T11:57:03.809 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:03.824+0000 7ff09fb57700 1 -- 172.21.15.93:0/4118854522 wait complete. 2023-07-14T11:57:03.809 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:03.825+0000 7ff09fb57700 1 Processor -- start 2023-07-14T11:57:03.809 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:03.825+0000 7ff09fb57700 1 -- start start 2023-07-14T11:57:03.809 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:03.825+0000 7ff09fb57700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0980ffb90 0x7ff0981959d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:03.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:03.825+0000 7ff09fb57700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff098100d90 0x7ff098195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:03.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:03.825+0000 7ff09fb57700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff098101720 0x7ff09819cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:03.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:03.825+0000 7ff09fb57700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09819d560 con 0x7ff098101720 2023-07-14T11:57:03.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:03.825+0000 7ff09fb57700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09819d6a0 con 0x7ff0980ffb90 2023-07-14T11:57:03.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:03.825+0000 7ff09fb57700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09819d830 con 0x7ff098100d90 2023-07-14T11:57:03.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:03.825+0000 7ff09d8f3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0980ffb90 0x7ff0981959d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:03.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:03.825+0000 7ff09d8f3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0980ffb90 0x7ff0981959d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:57786/0 (socket says 172.21.15.93:57786) 2023-07-14T11:57:03.811 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:03.825+0000 7ff09d8f3700 1 -- 172.21.15.93:0/2416857553 learned_addr learned my addr 172.21.15.93:0/2416857553 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-14T11:57:03.811 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:03.825+0000 7ff09d0f2700 1 --2- 172.21.15.93:0/2416857553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff098100d90 0x7ff098195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:03.811 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:03.825+0000 7ff09e0f4700 1 --2- 172.21.15.93:0/2416857553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff098101720 0x7ff09819cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:03.811 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:03.825+0000 7ff09d8f3700 1 -- 172.21.15.93:0/2416857553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff098100d90 msgr2=0x7ff098195f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:03.811 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:03.825+0000 7ff09d8f3700 1 --2- 172.21.15.93:0/2416857553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff098100d90 0x7ff098195f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:03.811 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:03.825+0000 7ff09d8f3700 1 -- 172.21.15.93:0/2416857553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff098101720 msgr2=0x7ff09819cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:03.811 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:03.825+0000 7ff09d8f3700 1 --2- 172.21.15.93:0/2416857553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff098101720 0x7ff09819cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:03.812 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:03.825+0000 7ff09d8f3700 1 -- 172.21.15.93:0/2416857553 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0940095f0 con 0x7ff0980ffb90 2023-07-14T11:57:03.812 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:03.826+0000 7ff09d8f3700 1 --2- 172.21.15.93:0/2416857553 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0980ffb90 0x7ff0981959d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00b500 tx=0x7ff08c00b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:03.812 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:03.826+0000 7ff08affd700 1 -- 172.21.15.93:0/2416857553 <== mon.2 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c011830 con 0x7ff0980ffb90 2023-07-14T11:57:03.812 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:03.826+0000 7ff08affd700 1 -- 172.21.15.93:0/2416857553 <== mon.2 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff08c011e50 con 0x7ff0980ffb90 2023-07-14T11:57:03.812 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:03.826+0000 7ff09fb57700 1 -- 172.21.15.93:0/2416857553 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff09819db10 con 0x7ff0980ffb90 2023-07-14T11:57:03.812 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:03.826+0000 7ff08affd700 1 -- 172.21.15.93:0/2416857553 <== mon.2 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c00f2e0 con 0x7ff0980ffb90 2023-07-14T11:57:03.812 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:03.826+0000 7ff09fb57700 1 -- 172.21.15.93:0/2416857553 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff09819e010 con 0x7ff0980ffb90 2023-07-14T11:57:03.813 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:03.827+0000 7ff09fb57700 1 -- 172.21.15.93:0/2416857553 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff098107180 con 0x7ff0980ffb90 2023-07-14T11:57:03.813 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:03.828+0000 7ff08affd700 1 -- 172.21.15.93:0/2416857553 <== mon.2 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff08c010580 con 0x7ff0980ffb90 2023-07-14T11:57:03.814 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:03.829+0000 7ff08affd700 1 --2- 172.21.15.93:0/2416857553 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7ff0840a1150 0x7ff0840a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:03.814 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:03.829+0000 7ff08affd700 1 -- 172.21.15.93:0/2416857553 <== mon.2 v2:172.21.15.93:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 4965+0+0 (secure 0 0 0) 0x7ff08c0cbc40 con 0x7ff0980ffb90 2023-07-14T11:57:03.814 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:03.829+0000 7ff09d0f2700 1 --2- 172.21.15.93:0/2416857553 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7ff0840a1150 0x7ff0840a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:03.814 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:03.829+0000 7ff09d0f2700 1 --2- 172.21.15.93:0/2416857553 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7ff0840a1150 0x7ff0840a3600 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff094006010 tx=0x7ff09400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:03.814 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:03.832+0000 7ff08affd700 1 -- 172.21.15.93:0/2416857553 <== mon.2 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff08c0960a0 con 0x7ff0980ffb90 2023-07-14T11:57:04.008 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:04.026+0000 7ff09fb57700 1 -- 172.21.15.93:0/2416857553 --> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7ff098060950 con 0x7ff0840a1150 2023-07-14T11:57:05.015 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:04 smithi093 ceph-mon[128922]: osdmap e48: 7 total, 7 up, 7 in 2023-07-14T11:57:05.016 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:04 smithi093 ceph-mon[128922]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:57:05.016 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:04 smithi093 ceph-mon[128922]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:57:05.016 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:04 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:57:05.016 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:04 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:57:05.016 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:04 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:57:05.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:04 smithi097 ceph-mon[129313]: osdmap e48: 7 total, 7 up, 7 in 2023-07-14T11:57:05.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:04 smithi097 ceph-mon[129313]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:57:05.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:04 smithi097 ceph-mon[129313]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:57:05.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:04 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:57:05.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:04 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:57:05.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:04 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:57:05.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:04 smithi017 ceph-mon[126339]: osdmap e48: 7 total, 7 up, 7 in 2023-07-14T11:57:05.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:04 smithi017 ceph-mon[126339]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:57:05.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:04 smithi017 ceph-mon[126339]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:57:05.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:04 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:57:05.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:04 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:57:05.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:04 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:57:07.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:06 smithi093 ceph-mon[128922]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:57:07.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:06 smithi093 ceph-mon[128922]: from='client.? 172.21.15.93:0/528156340' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09f60823-5491-45cd-9324-6ae0d56ce262"}]: dispatch 2023-07-14T11:57:07.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:06 smithi093 ceph-mon[128922]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09f60823-5491-45cd-9324-6ae0d56ce262"}]: dispatch 2023-07-14T11:57:07.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:06 smithi093 ceph-mon[128922]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09f60823-5491-45cd-9324-6ae0d56ce262"}]': finished 2023-07-14T11:57:07.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:06 smithi093 ceph-mon[128922]: osdmap e49: 8 total, 7 up, 8 in 2023-07-14T11:57:07.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:06 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T11:57:07.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:06 smithi097 ceph-mon[129313]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:57:07.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:06 smithi097 ceph-mon[129313]: from='client.? 172.21.15.93:0/528156340' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09f60823-5491-45cd-9324-6ae0d56ce262"}]: dispatch 2023-07-14T11:57:07.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:06 smithi097 ceph-mon[129313]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09f60823-5491-45cd-9324-6ae0d56ce262"}]: dispatch 2023-07-14T11:57:07.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:06 smithi097 ceph-mon[129313]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09f60823-5491-45cd-9324-6ae0d56ce262"}]': finished 2023-07-14T11:57:07.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:06 smithi097 ceph-mon[129313]: osdmap e49: 8 total, 7 up, 8 in 2023-07-14T11:57:07.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:06 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T11:57:07.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:06 smithi017 ceph-mon[126339]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:57:07.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:06 smithi017 ceph-mon[126339]: from='client.? 172.21.15.93:0/528156340' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09f60823-5491-45cd-9324-6ae0d56ce262"}]: dispatch 2023-07-14T11:57:07.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:06 smithi017 ceph-mon[126339]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09f60823-5491-45cd-9324-6ae0d56ce262"}]: dispatch 2023-07-14T11:57:07.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:06 smithi017 ceph-mon[126339]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09f60823-5491-45cd-9324-6ae0d56ce262"}]': finished 2023-07-14T11:57:07.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:06 smithi017 ceph-mon[126339]: osdmap e49: 8 total, 7 up, 8 in 2023-07-14T11:57:07.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:06 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T11:57:08.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:07 smithi093 ceph-mon[128922]: from='client.? 172.21.15.93:0/3473120014' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:57:08.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:07 smithi097 ceph-mon[129313]: from='client.? 172.21.15.93:0/3473120014' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:57:08.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:07 smithi017 ceph-mon[126339]: from='client.? 172.21.15.93:0/3473120014' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:57:09.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:08 smithi093 ceph-mon[128922]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:57:09.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:08 smithi097 ceph-mon[129313]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:57:09.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:08 smithi017 ceph-mon[126339]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:57:11.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:10 smithi093 ceph-mon[128922]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:57:11.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:10 smithi097 ceph-mon[129313]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:57:11.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:10 smithi017 ceph-mon[126339]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:57:13.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:12 smithi093 ceph-mon[128922]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:57:13.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:12 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-07-14T11:57:13.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:12 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:57:13.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:12 smithi097 ceph-mon[129313]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:57:13.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:12 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-07-14T11:57:13.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:12 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:57:13.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:12 smithi017 ceph-mon[126339]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:57:13.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:12 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-07-14T11:57:13.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:12 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:57:13.862 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:13 smithi093 ceph-mon[128922]: Deploying daemon osd.7 on smithi093 2023-07-14T11:57:14.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:13 smithi097 ceph-mon[129313]: Deploying daemon osd.7 on smithi093 2023-07-14T11:57:14.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:13 smithi017 ceph-mon[126339]: Deploying daemon osd.7 on smithi093 2023-07-14T11:57:14.862 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:14 smithi093 ceph-mon[128922]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:57:15.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:14 smithi097 ceph-mon[129313]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:57:15.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:14 smithi017 ceph-mon[126339]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:57:17.018 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:16 smithi093 ceph-mon[128922]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:57:17.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:16 smithi097 ceph-mon[129313]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:57:17.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:16 smithi017 ceph-mon[126339]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:57:18.313 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:18 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:18.314 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:18 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:18.314 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:18 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:57:18.314 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:18 smithi093 ceph-mon[128922]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:57:18.635 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:18 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:18.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:18 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:18.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:18 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:57:18.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:18 smithi097 ceph-mon[129313]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:57:18.645 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:18 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:18.645 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:18 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:18.645 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:18 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:57:18.645 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:18 smithi017 ceph-mon[126339]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:57:19.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:19.990+0000 7ff08affd700 1 -- 172.21.15.93:0/2416857553 <== mgr.14152 v2:172.21.15.17:6800/4237449858 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff098060950 con 0x7ff0840a1150 2023-07-14T11:57:19.972 INFO:teuthology.orchestra.run.smithi093.stdout:Created osd(s) 7 on host 'smithi093' 2023-07-14T11:57:19.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:19.993+0000 7ff09fb57700 1 -- 172.21.15.93:0/2416857553 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7ff0840a1150 msgr2=0x7ff0840a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:19.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:19.993+0000 7ff09fb57700 1 --2- 172.21.15.93:0/2416857553 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7ff0840a1150 0x7ff0840a3600 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff094006010 tx=0x7ff09400b540 comp rx=0 tx=0).stop 2023-07-14T11:57:19.975 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:19.993+0000 7ff09fb57700 1 -- 172.21.15.93:0/2416857553 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0980ffb90 msgr2=0x7ff0981959d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:19.975 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:19.993+0000 7ff09fb57700 1 --2- 172.21.15.93:0/2416857553 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0980ffb90 0x7ff0981959d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00b500 tx=0x7ff08c00b8c0 comp rx=0 tx=0).stop 2023-07-14T11:57:19.975 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:19.993+0000 7ff09fb57700 1 -- 172.21.15.93:0/2416857553 shutdown_connections 2023-07-14T11:57:19.976 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:19.993+0000 7ff09fb57700 1 --2- 172.21.15.93:0/2416857553 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7ff0840a1150 0x7ff0840a3600 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:19.976 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:19.993+0000 7ff09fb57700 1 --2- 172.21.15.93:0/2416857553 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0980ffb90 0x7ff0981959d0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:19.976 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:19.993+0000 7ff09fb57700 1 --2- 172.21.15.93:0/2416857553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff098101720 0x7ff09819cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:19.976 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:19.993+0000 7ff09fb57700 1 --2- 172.21.15.93:0/2416857553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff098100d90 0x7ff098195f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:19.977 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:19.993+0000 7ff09fb57700 1 -- 172.21.15.93:0/2416857553 >> 172.21.15.93:0/2416857553 conn(0x7ff0980fb1a0 msgr2=0x7ff0981042c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:19.977 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:19.993+0000 7ff09fb57700 1 -- 172.21.15.93:0/2416857553 shutdown_connections 2023-07-14T11:57:19.977 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:57:19.993+0000 7ff09fb57700 1 -- 172.21.15.93:0/2416857553 wait complete. 2023-07-14T11:57:20.397 DEBUG:teuthology.orchestra.run.smithi093:osd.7> sudo journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@osd.7.service 2023-07-14T11:57:20.399 INFO:tasks.cephadm:Deploying osd.8 on smithi097 with /dev/vg_nvme/lv_2... 2023-07-14T11:57:20.400 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-07-14T11:57:20.432 INFO:journalctl@ceph.osd.7.smithi093.stdout:-- Logs begin at Fri 2023-07-14 11:24:16 UTC. -- 2023-07-14T11:57:20.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:20 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:20.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:20 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:20.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:20 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:57:20.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:20 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:57:20.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:20 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:20.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:20 smithi097 ceph-mon[129313]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:57:20.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:20 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:20.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:20 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:20.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:20 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:57:20.916 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:20 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:20.917 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:20 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:20.917 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:20 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:57:20.917 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:20 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:57:20.917 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:20 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:20.917 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:20 smithi093 ceph-mon[128922]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:57:20.917 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:20 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:20.918 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:20 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:20.918 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:20 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:57:20.962 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T11:57:21.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:20 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:21.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:20 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:21.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:20 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:57:21.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:20 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:57:21.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:20 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:21.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:20 smithi017 ceph-mon[126339]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:57:21.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:20 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:21.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:20 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:21.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:20 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:57:21.362 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 11:57:21 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T11:57:21.089+0000 7f2fbd2ee640 -1 osd.7 0 log_to_monitors true 2023-07-14T11:57:21.807 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:21 smithi097 ceph-mon[129313]: from='osd.7 [v2:172.21.15.93:6816/2596912392,v1:172.21.15.93:6817/2596912392]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-14T11:57:21.808 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:21 smithi097 ceph-mon[129313]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-14T11:57:22.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:21 smithi093 ceph-mon[128922]: from='osd.7 [v2:172.21.15.93:6816/2596912392,v1:172.21.15.93:6817/2596912392]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-14T11:57:22.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:21 smithi093 ceph-mon[128922]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-14T11:57:22.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:21 smithi017 ceph-mon[126339]: from='osd.7 [v2:172.21.15.93:6816/2596912392,v1:172.21.15.93:6817/2596912392]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-14T11:57:22.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:21 smithi017 ceph-mon[126339]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-14T11:57:22.592 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-07-14T11:57:22.593 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-07-14T11:57:22.593 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2023-07-14T11:57:22.593 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2023-07-14T11:57:22.593 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110647 s, 948 MB/s 2023-07-14T11:57:22.593 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2023-07-14T11:57:22.862 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 11:57:22 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T11:57:22.699+0000 7f2fb2966700 -1 osd.7 0 waiting for initial osdmap 2023-07-14T11:57:22.862 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 11:57:22 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T11:57:22.711+0000 7f2fae75b700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T11:57:22.862 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:22 smithi093 ceph-mon[128922]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-07-14T11:57:22.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:22 smithi093 ceph-mon[128922]: osdmap e50: 8 total, 7 up, 8 in 2023-07-14T11:57:22.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:22 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T11:57:22.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:22 smithi093 ceph-mon[128922]: from='osd.7 [v2:172.21.15.93:6816/2596912392,v1:172.21.15.93:6817/2596912392]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-07-14T11:57:22.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:22 smithi093 ceph-mon[128922]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-07-14T11:57:22.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:22 smithi093 ceph-mon[128922]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:57:22.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:22 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:22.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:22 smithi097 ceph-mon[129313]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-07-14T11:57:22.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:22 smithi097 ceph-mon[129313]: osdmap e50: 8 total, 7 up, 8 in 2023-07-14T11:57:22.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:22 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T11:57:22.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:22 smithi097 ceph-mon[129313]: from='osd.7 [v2:172.21.15.93:6816/2596912392,v1:172.21.15.93:6817/2596912392]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-07-14T11:57:22.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:22 smithi097 ceph-mon[129313]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-07-14T11:57:22.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:22 smithi097 ceph-mon[129313]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:57:22.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:22 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:23.007 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi097:vg_nvme/lv_2 2023-07-14T11:57:23.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:22 smithi017 ceph-mon[126339]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-07-14T11:57:23.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:22 smithi017 ceph-mon[126339]: osdmap e50: 8 total, 7 up, 8 in 2023-07-14T11:57:23.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:22 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T11:57:23.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:22 smithi017 ceph-mon[126339]: from='osd.7 [v2:172.21.15.93:6816/2596912392,v1:172.21.15.93:6817/2596912392]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-07-14T11:57:23.148 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:22 smithi017 ceph-mon[126339]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-07-14T11:57:23.148 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:22 smithi017 ceph-mon[126339]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:57:23.149 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:22 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:23.555 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T11:57:24.036 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:23 smithi097 ceph-mon[129313]: Detected new or changed devices on smithi093 2023-07-14T11:57:24.036 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:23 smithi097 ceph-mon[129313]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-07-14T11:57:24.036 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:23 smithi097 ceph-mon[129313]: osdmap e51: 8 total, 7 up, 8 in 2023-07-14T11:57:24.036 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:23 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T11:57:24.037 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:23 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:24.037 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:23 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T11:57:24.037 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:23 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:57:24.037 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:23 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:57:24.037 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:23 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:57:24.037 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:23 smithi097 ceph-mon[129313]: Adjusting osd_memory_target on smithi093 to 5722M 2023-07-14T11:57:24.037 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:23 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:24.038 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:23 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:57:24.038 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:23 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:57:24.038 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:23 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:24.038 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:23 smithi097 ceph-mon[129313]: osd.7 [v2:172.21.15.93:6816/2596912392,v1:172.21.15.93:6817/2596912392] boot 2023-07-14T11:57:24.038 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:23 smithi097 ceph-mon[129313]: osdmap e52: 8 total, 8 up, 8 in 2023-07-14T11:57:24.038 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:23 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T11:57:24.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:23 smithi093 ceph-mon[128922]: Detected new or changed devices on smithi093 2023-07-14T11:57:24.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:23 smithi093 ceph-mon[128922]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-07-14T11:57:24.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:23 smithi093 ceph-mon[128922]: osdmap e51: 8 total, 7 up, 8 in 2023-07-14T11:57:24.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:23 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T11:57:24.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:23 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:24.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:23 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T11:57:24.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:23 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:57:24.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:23 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:57:24.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:23 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:57:24.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:23 smithi093 ceph-mon[128922]: Adjusting osd_memory_target on smithi093 to 5722M 2023-07-14T11:57:24.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:23 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:24.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:23 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:57:24.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:23 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:57:24.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:23 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:24.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:23 smithi093 ceph-mon[128922]: osd.7 [v2:172.21.15.93:6816/2596912392,v1:172.21.15.93:6817/2596912392] boot 2023-07-14T11:57:24.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:23 smithi093 ceph-mon[128922]: osdmap e52: 8 total, 8 up, 8 in 2023-07-14T11:57:24.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:23 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T11:57:24.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:23 smithi017 ceph-mon[126339]: Detected new or changed devices on smithi093 2023-07-14T11:57:24.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:23 smithi017 ceph-mon[126339]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-07-14T11:57:24.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:23 smithi017 ceph-mon[126339]: osdmap e51: 8 total, 7 up, 8 in 2023-07-14T11:57:24.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:23 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T11:57:24.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:23 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:24.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:23 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T11:57:24.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:23 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:57:24.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:23 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:57:24.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:23 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:57:24.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:23 smithi017 ceph-mon[126339]: Adjusting osd_memory_target on smithi093 to 5722M 2023-07-14T11:57:24.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:23 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:24.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:23 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:57:24.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:23 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:57:24.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:23 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:24.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:23 smithi017 ceph-mon[126339]: osd.7 [v2:172.21.15.93:6816/2596912392,v1:172.21.15.93:6817/2596912392] boot 2023-07-14T11:57:24.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:23 smithi017 ceph-mon[126339]: osdmap e52: 8 total, 8 up, 8 in 2023-07-14T11:57:24.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:23 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T11:57:24.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:24.783+0000 7fdc87598700 1 -- 172.21.15.97:0/3713814552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc800fed20 msgr2=0x7fdc800ff170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:24.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:24.783+0000 7fdc87598700 1 --2- 172.21.15.97:0/3713814552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc800fed20 0x7fdc800ff170 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c009990 tx=0x7fdc7c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:57:24.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:24.783+0000 7fdc87598700 1 -- 172.21.15.97:0/3713814552 shutdown_connections 2023-07-14T11:57:24.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:24.783+0000 7fdc87598700 1 --2- 172.21.15.97:0/3713814552 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdc800ff6b0 0x7fdc80106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:24.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:24.783+0000 7fdc87598700 1 --2- 172.21.15.97:0/3713814552 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc80100070 0x7fdc80100480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:24.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:24.783+0000 7fdc87598700 1 --2- 172.21.15.97:0/3713814552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc800fed20 0x7fdc800ff170 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:24.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:24.784+0000 7fdc87598700 1 -- 172.21.15.97:0/3713814552 >> 172.21.15.97:0/3713814552 conn(0x7fdc800f9860 msgr2=0x7fdc800fbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:24.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:24.784+0000 7fdc87598700 1 -- 172.21.15.97:0/3713814552 shutdown_connections 2023-07-14T11:57:24.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:24.784+0000 7fdc87598700 1 -- 172.21.15.97:0/3713814552 wait complete. 2023-07-14T11:57:24.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:24.784+0000 7fdc87598700 1 Processor -- start 2023-07-14T11:57:24.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:24.784+0000 7fdc87598700 1 -- start start 2023-07-14T11:57:24.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:24.784+0000 7fdc87598700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc800fed20 0x7fdc80102ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:24.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:24.784+0000 7fdc87598700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc800ff6b0 0x7fdc80101110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:24.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:24.784+0000 7fdc87598700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdc80100070 0x7fdc80101650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:24.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:24.784+0000 7fdc87598700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc80101dd0 con 0x7fdc800fed20 2023-07-14T11:57:24.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:24.784+0000 7fdc87598700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc80101f40 con 0x7fdc80100070 2023-07-14T11:57:24.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:24.784+0000 7fdc87598700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8019f920 con 0x7fdc800ff6b0 2023-07-14T11:57:24.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:24.784+0000 7fdc84b33700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc800ff6b0 0x7fdc80101110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:24.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:24.784+0000 7fdc84b33700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc800ff6b0 0x7fdc80101110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42412/0 (socket says 172.21.15.97:42412) 2023-07-14T11:57:24.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:24.784+0000 7fdc84b33700 1 -- 172.21.15.97:0/3870803088 learned_addr learned my addr 172.21.15.97:0/3870803088 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-14T11:57:24.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:24.785+0000 7fdc85b35700 1 --2- 172.21.15.97:0/3870803088 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdc80100070 0x7fdc80101650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:24.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:24.785+0000 7fdc84b33700 1 -- 172.21.15.97:0/3870803088 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdc80100070 msgr2=0x7fdc80101650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:24.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:24.785+0000 7fdc84b33700 1 --2- 172.21.15.97:0/3870803088 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdc80100070 0x7fdc80101650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:24.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:24.785+0000 7fdc84b33700 1 -- 172.21.15.97:0/3870803088 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc800fed20 msgr2=0x7fdc80102ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:24.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:24.785+0000 7fdc84b33700 1 --2- 172.21.15.97:0/3870803088 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc800fed20 0x7fdc80102ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:24.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:24.785+0000 7fdc84b33700 1 -- 172.21.15.97:0/3870803088 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc7c0095f0 con 0x7fdc800ff6b0 2023-07-14T11:57:24.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:24.785+0000 7fdc84b33700 1 --2- 172.21.15.97:0/3870803088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc800ff6b0 0x7fdc80101110 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c000c00 tx=0x7fdc7c00f6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:24.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:24.785+0000 7fdc767fc700 1 -- 172.21.15.97:0/3870803088 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c01d070 con 0x7fdc800ff6b0 2023-07-14T11:57:24.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:24.785+0000 7fdc87598700 1 -- 172.21.15.97:0/3870803088 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc8019fba0 con 0x7fdc800ff6b0 2023-07-14T11:57:24.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:24.785+0000 7fdc767fc700 1 -- 172.21.15.97:0/3870803088 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc7c00fb80 con 0x7fdc800ff6b0 2023-07-14T11:57:24.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:24.785+0000 7fdc767fc700 1 -- 172.21.15.97:0/3870803088 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c017650 con 0x7fdc800ff6b0 2023-07-14T11:57:24.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:24.785+0000 7fdc87598700 1 -- 172.21.15.97:0/3870803088 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc8019ffb0 con 0x7fdc800ff6b0 2023-07-14T11:57:24.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:24.786+0000 7fdc87598700 1 -- 172.21.15.97:0/3870803088 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc8010a2f0 con 0x7fdc800ff6b0 2023-07-14T11:57:24.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:24.787+0000 7fdc767fc700 1 -- 172.21.15.97:0/3870803088 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fdc7c022470 con 0x7fdc800ff6b0 2023-07-14T11:57:24.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:24.788+0000 7fdc767fc700 1 --2- 172.21.15.97:0/3870803088 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fdc6c0a1150 0x7fdc6c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:24.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:24.788+0000 7fdc767fc700 1 -- 172.21.15.97:0/3870803088 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5426+0+0 (secure 0 0 0) 0x7fdc7c0cd5b0 con 0x7fdc800ff6b0 2023-07-14T11:57:24.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:24.788+0000 7fdc85334700 1 --2- 172.21.15.97:0/3870803088 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fdc6c0a1150 0x7fdc6c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:24.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:24.789+0000 7fdc85334700 1 --2- 172.21.15.97:0/3870803088 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fdc6c0a1150 0x7fdc6c0a3600 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdc80067d90 tx=0x7fdc70006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:24.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:24.791+0000 7fdc767fc700 1 -- 172.21.15.97:0/3870803088 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fdc7c0978d0 con 0x7fdc800ff6b0 2023-07-14T11:57:24.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:24 smithi097 ceph-mon[129313]: purged_snaps scrub starts 2023-07-14T11:57:24.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:24 smithi097 ceph-mon[129313]: purged_snaps scrub ok 2023-07-14T11:57:24.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:24 smithi097 ceph-mon[129313]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:57:24.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:24 smithi097 ceph-mon[129313]: osdmap e53: 8 total, 8 up, 8 in 2023-07-14T11:57:24.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:24.986+0000 7fdc87598700 1 -- 172.21.15.97:0/3870803088 --> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc800fe3c0 con 0x7fdc6c0a1150 2023-07-14T11:57:25.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:24 smithi093 ceph-mon[128922]: purged_snaps scrub starts 2023-07-14T11:57:25.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:24 smithi093 ceph-mon[128922]: purged_snaps scrub ok 2023-07-14T11:57:25.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:24 smithi093 ceph-mon[128922]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:57:25.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:24 smithi093 ceph-mon[128922]: osdmap e53: 8 total, 8 up, 8 in 2023-07-14T11:57:25.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:24 smithi017 ceph-mon[126339]: purged_snaps scrub starts 2023-07-14T11:57:25.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:24 smithi017 ceph-mon[126339]: purged_snaps scrub ok 2023-07-14T11:57:25.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:24 smithi017 ceph-mon[126339]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T11:57:25.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:24 smithi017 ceph-mon[126339]: osdmap e53: 8 total, 8 up, 8 in 2023-07-14T11:57:26.027 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:25 smithi097 ceph-mon[129313]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:57:26.027 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:25 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:57:26.028 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:25 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:57:26.028 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:25 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:57:26.028 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:25 smithi097 ceph-mon[129313]: osdmap e54: 8 total, 8 up, 8 in 2023-07-14T11:57:26.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:25 smithi093 ceph-mon[128922]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:57:26.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:25 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:57:26.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:25 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:57:26.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:25 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:57:26.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:25 smithi093 ceph-mon[128922]: osdmap e54: 8 total, 8 up, 8 in 2023-07-14T11:57:26.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:25 smithi017 ceph-mon[126339]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:57:26.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:25 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:57:26.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:25 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:57:26.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:25 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:57:26.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:25 smithi017 ceph-mon[126339]: osdmap e54: 8 total, 8 up, 8 in 2023-07-14T11:57:27.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:26 smithi093 ceph-mon[128922]: pgmap v174: 1 pgs: 1 remapped+peering; 577 KiB data, 214 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:57:27.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:26 smithi097 ceph-mon[129313]: pgmap v174: 1 pgs: 1 remapped+peering; 577 KiB data, 214 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:57:27.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:26 smithi017 ceph-mon[126339]: pgmap v174: 1 pgs: 1 remapped+peering; 577 KiB data, 214 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:57:28.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:27 smithi093 ceph-mon[128922]: osdmap e55: 8 total, 8 up, 8 in 2023-07-14T11:57:28.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:27 smithi093 ceph-mon[128922]: from='client.? 172.21.15.97:0/4026087845' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c5771c9-718e-4313-a2b6-5e1c37c53f30"}]: dispatch 2023-07-14T11:57:28.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:27 smithi093 ceph-mon[128922]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c5771c9-718e-4313-a2b6-5e1c37c53f30"}]: dispatch 2023-07-14T11:57:28.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:27 smithi093 ceph-mon[128922]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8c5771c9-718e-4313-a2b6-5e1c37c53f30"}]': finished 2023-07-14T11:57:28.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:27 smithi093 ceph-mon[128922]: osdmap e56: 9 total, 8 up, 9 in 2023-07-14T11:57:28.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:27 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T11:57:28.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:27 smithi097 ceph-mon[129313]: osdmap e55: 8 total, 8 up, 8 in 2023-07-14T11:57:28.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:27 smithi097 ceph-mon[129313]: from='client.? 172.21.15.97:0/4026087845' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c5771c9-718e-4313-a2b6-5e1c37c53f30"}]: dispatch 2023-07-14T11:57:28.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:27 smithi097 ceph-mon[129313]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c5771c9-718e-4313-a2b6-5e1c37c53f30"}]: dispatch 2023-07-14T11:57:28.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:27 smithi097 ceph-mon[129313]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8c5771c9-718e-4313-a2b6-5e1c37c53f30"}]': finished 2023-07-14T11:57:28.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:27 smithi097 ceph-mon[129313]: osdmap e56: 9 total, 8 up, 9 in 2023-07-14T11:57:28.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:27 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T11:57:28.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:27 smithi017 ceph-mon[126339]: osdmap e55: 8 total, 8 up, 8 in 2023-07-14T11:57:28.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:27 smithi017 ceph-mon[126339]: from='client.? 172.21.15.97:0/4026087845' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c5771c9-718e-4313-a2b6-5e1c37c53f30"}]: dispatch 2023-07-14T11:57:28.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:27 smithi017 ceph-mon[126339]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c5771c9-718e-4313-a2b6-5e1c37c53f30"}]: dispatch 2023-07-14T11:57:28.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:27 smithi017 ceph-mon[126339]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8c5771c9-718e-4313-a2b6-5e1c37c53f30"}]': finished 2023-07-14T11:57:28.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:27 smithi017 ceph-mon[126339]: osdmap e56: 9 total, 8 up, 9 in 2023-07-14T11:57:28.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:27 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T11:57:29.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:28 smithi093 ceph-mon[128922]: pgmap v177: 1 pgs: 1 remapped+peering; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:57:29.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:28 smithi093 ceph-mon[128922]: from='client.? 172.21.15.97:0/303856285' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:57:29.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:28 smithi097 ceph-mon[129313]: pgmap v177: 1 pgs: 1 remapped+peering; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:57:29.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:28 smithi097 ceph-mon[129313]: from='client.? 172.21.15.97:0/303856285' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:57:29.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:28 smithi017 ceph-mon[126339]: pgmap v177: 1 pgs: 1 remapped+peering; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:57:29.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:28 smithi017 ceph-mon[126339]: from='client.? 172.21.15.97:0/303856285' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:57:30.999 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:30 smithi097 ceph-mon[129313]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:57:31.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:30 smithi093 ceph-mon[128922]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:57:31.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:30 smithi017 ceph-mon[126339]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:57:32.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:32 smithi097 ceph-mon[129313]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:57:33.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:32 smithi093 ceph-mon[128922]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:57:33.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:32 smithi017 ceph-mon[126339]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:57:34.645 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:34 smithi017 ceph-mon[126339]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:57:34.645 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:34 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-07-14T11:57:34.645 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:34 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:57:34.646 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:34 smithi017 ceph-mon[126339]: Deploying daemon osd.8 on smithi097 2023-07-14T11:57:34.792 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:34 smithi097 ceph-mon[129313]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:57:34.792 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:34 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-07-14T11:57:34.793 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:34 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:57:34.793 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:34 smithi097 ceph-mon[129313]: Deploying daemon osd.8 on smithi097 2023-07-14T11:57:34.862 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:34 smithi093 ceph-mon[128922]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:57:34.862 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:34 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-07-14T11:57:34.862 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:34 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:57:34.862 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:34 smithi093 ceph-mon[128922]: Deploying daemon osd.8 on smithi097 2023-07-14T11:57:37.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:36 smithi093 ceph-mon[128922]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:57:37.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:36 smithi097 ceph-mon[129313]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:57:37.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:36 smithi017 ceph-mon[126339]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:57:39.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:38 smithi093 ceph-mon[128922]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:57:39.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:38 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:39.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:38 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:57:39.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:38 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:39.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:38 smithi097 ceph-mon[129313]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:57:39.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:38 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:39.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:38 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:57:39.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:38 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:39.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:38 smithi017 ceph-mon[126339]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:57:39.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:38 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:39.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:38 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:57:39.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:38 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:41.100 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:40 smithi097 ceph-mon[129313]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:57:41.100 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:40 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:41.100 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:40 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:41.100 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:40 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:57:41.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:40 smithi093 ceph-mon[128922]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:57:41.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:40 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:41.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:40 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:41.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:40 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:57:41.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:40 smithi017 ceph-mon[126339]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:57:41.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:40 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:41.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:40 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:41.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:40 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:57:41.433 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 8 on host 'smithi097' 2023-07-14T11:57:41.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:41.451+0000 7fdc767fc700 1 -- 172.21.15.97:0/3870803088 <== mgr.14152 v2:172.21.15.17:6800/4237449858 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fdc800fe3c0 con 0x7fdc6c0a1150 2023-07-14T11:57:41.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:41.453+0000 7fdc87598700 1 -- 172.21.15.97:0/3870803088 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fdc6c0a1150 msgr2=0x7fdc6c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:41.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:41.453+0000 7fdc87598700 1 --2- 172.21.15.97:0/3870803088 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fdc6c0a1150 0x7fdc6c0a3600 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdc80067d90 tx=0x7fdc70006d20 comp rx=0 tx=0).stop 2023-07-14T11:57:41.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:41.453+0000 7fdc87598700 1 -- 172.21.15.97:0/3870803088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc800ff6b0 msgr2=0x7fdc80101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:41.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:41.453+0000 7fdc87598700 1 --2- 172.21.15.97:0/3870803088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc800ff6b0 0x7fdc80101110 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c000c00 tx=0x7fdc7c00f6e0 comp rx=0 tx=0).stop 2023-07-14T11:57:41.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:41.454+0000 7fdc87598700 1 -- 172.21.15.97:0/3870803088 shutdown_connections 2023-07-14T11:57:41.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:41.454+0000 7fdc87598700 1 --2- 172.21.15.97:0/3870803088 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fdc6c0a1150 0x7fdc6c0a3600 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:41.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:41.454+0000 7fdc87598700 1 --2- 172.21.15.97:0/3870803088 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdc80100070 0x7fdc80101650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:41.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:41.454+0000 7fdc87598700 1 --2- 172.21.15.97:0/3870803088 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc800fed20 0x7fdc80102ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:41.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:41.454+0000 7fdc87598700 1 --2- 172.21.15.97:0/3870803088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc800ff6b0 0x7fdc80101110 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:41.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:41.454+0000 7fdc87598700 1 -- 172.21.15.97:0/3870803088 >> 172.21.15.97:0/3870803088 conn(0x7fdc800f9860 msgr2=0x7fdc800fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:41.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:41.454+0000 7fdc87598700 1 -- 172.21.15.97:0/3870803088 shutdown_connections 2023-07-14T11:57:41.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:57:41.454+0000 7fdc87598700 1 -- 172.21.15.97:0/3870803088 wait complete. 2023-07-14T11:57:41.945 DEBUG:teuthology.orchestra.run.smithi097:osd.8> sudo journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@osd.8.service 2023-07-14T11:57:41.949 INFO:tasks.cephadm:Deploying osd.9 on smithi017 with /dev/vg_nvme/lv_1... 2023-07-14T11:57:41.950 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-07-14T11:57:41.981 INFO:journalctl@ceph.osd.8.smithi097.stdout:-- Logs begin at Fri 2023-07-14 11:24:16 UTC. -- 2023-07-14T11:57:42.020 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:41 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:57:42.021 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:41 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:42.021 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:41 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:42.021 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:41 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:42.021 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:41 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:57:42.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:41 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:57:42.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:41 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:42.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:41 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:42.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:41 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:42.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:41 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:57:42.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:41 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:57:42.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:41 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:42.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:41 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:42.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:41 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:42.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:41 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:57:42.600 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 11:57:42 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T11:57:42.230+0000 7ff87040a640 -1 osd.8 0 log_to_monitors true 2023-07-14T11:57:42.693 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T11:57:42.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:42 smithi097 ceph-mon[129313]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:57:42.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:42 smithi097 ceph-mon[129313]: from='osd.8 [v2:172.21.15.97:6816/1965492859,v1:172.21.15.97:6817/1965492859]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-14T11:57:42.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:42 smithi097 ceph-mon[129313]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-14T11:57:43.008 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:42 smithi017 ceph-mon[126339]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:57:43.008 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:42 smithi017 ceph-mon[126339]: from='osd.8 [v2:172.21.15.97:6816/1965492859,v1:172.21.15.97:6817/1965492859]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-14T11:57:43.009 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:42 smithi017 ceph-mon[126339]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-14T11:57:43.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:42 smithi093 ceph-mon[128922]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:57:43.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:42 smithi093 ceph-mon[128922]: from='osd.8 [v2:172.21.15.97:6816/1965492859,v1:172.21.15.97:6817/1965492859]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-14T11:57:43.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:42 smithi093 ceph-mon[128922]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-14T11:57:43.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:43 smithi017 ceph-mon[126339]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-07-14T11:57:43.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:43 smithi017 ceph-mon[126339]: osdmap e57: 9 total, 8 up, 9 in 2023-07-14T11:57:43.896 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:43 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T11:57:43.896 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:43 smithi017 ceph-mon[126339]: from='osd.8 [v2:172.21.15.97:6816/1965492859,v1:172.21.15.97:6817/1965492859]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-07-14T11:57:43.896 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:43 smithi017 ceph-mon[126339]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-07-14T11:57:44.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:43 smithi093 ceph-mon[128922]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-07-14T11:57:44.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:43 smithi093 ceph-mon[128922]: osdmap e57: 9 total, 8 up, 9 in 2023-07-14T11:57:44.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:43 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T11:57:44.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:43 smithi093 ceph-mon[128922]: from='osd.8 [v2:172.21.15.97:6816/1965492859,v1:172.21.15.97:6817/1965492859]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-07-14T11:57:44.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:43 smithi093 ceph-mon[128922]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-07-14T11:57:44.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:43 smithi097 ceph-mon[129313]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-07-14T11:57:44.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:43 smithi097 ceph-mon[129313]: osdmap e57: 9 total, 8 up, 9 in 2023-07-14T11:57:44.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:43 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T11:57:44.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:43 smithi097 ceph-mon[129313]: from='osd.8 [v2:172.21.15.97:6816/1965492859,v1:172.21.15.97:6817/1965492859]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-07-14T11:57:44.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:43 smithi097 ceph-mon[129313]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-07-14T11:57:44.137 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 11:57:43 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T11:57:43.774+0000 7ff865a82700 -1 osd.8 0 waiting for initial osdmap 2023-07-14T11:57:44.137 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 11:57:43 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T11:57:43.784+0000 7ff860074700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T11:57:44.704 INFO:teuthology.orchestra.run.smithi017.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-07-14T11:57:44.704 INFO:teuthology.orchestra.run.smithi017.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-07-14T11:57:44.705 INFO:teuthology.orchestra.run.smithi017.stderr: stderr: 10+0 records in 2023-07-14T11:57:44.705 INFO:teuthology.orchestra.run.smithi017.stderr:10+0 records out 2023-07-14T11:57:44.705 INFO:teuthology.orchestra.run.smithi017.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0388956 s, 270 MB/s 2023-07-14T11:57:44.705 INFO:teuthology.orchestra.run.smithi017.stderr:--> Zapping successful for: 2023-07-14T11:57:45.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:44 smithi093 ceph-mon[128922]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:57:45.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:44 smithi093 ceph-mon[128922]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2023-07-14T11:57:45.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:44 smithi093 ceph-mon[128922]: osdmap e58: 9 total, 8 up, 9 in 2023-07-14T11:57:45.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:44 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T11:57:45.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:44 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T11:57:45.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:44 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:45.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:44 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:45.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:44 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:57:45.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:44 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:57:45.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:44 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:57:45.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:44 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:45.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:44 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:57:45.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:44 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:57:45.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:44 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:45.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:44 smithi097 ceph-mon[129313]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:57:45.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:44 smithi097 ceph-mon[129313]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2023-07-14T11:57:45.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:44 smithi097 ceph-mon[129313]: osdmap e58: 9 total, 8 up, 9 in 2023-07-14T11:57:45.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:44 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T11:57:45.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:44 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T11:57:45.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:44 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:45.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:44 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:45.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:44 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:57:45.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:44 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:57:45.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:44 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:57:45.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:44 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:45.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:44 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:57:45.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:44 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:57:45.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:44 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:45.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:44 smithi017 ceph-mon[126339]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:57:45.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:44 smithi017 ceph-mon[126339]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2023-07-14T11:57:45.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:44 smithi017 ceph-mon[126339]: osdmap e58: 9 total, 8 up, 9 in 2023-07-14T11:57:45.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:44 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T11:57:45.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:44 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T11:57:45.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:44 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:45.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:44 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:45.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:44 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:57:45.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:44 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:57:45.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:44 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:57:45.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:44 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:45.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:44 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:57:45.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:44 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:57:45.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:44 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:57:45.271 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi017:vg_nvme/lv_1 2023-07-14T11:57:46.007 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T11:57:46.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:45 smithi093 ceph-mon[128922]: purged_snaps scrub starts 2023-07-14T11:57:46.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:45 smithi093 ceph-mon[128922]: purged_snaps scrub ok 2023-07-14T11:57:46.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:45 smithi093 ceph-mon[128922]: Detected new or changed devices on smithi097 2023-07-14T11:57:46.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:45 smithi093 ceph-mon[128922]: Adjusting osd_memory_target on smithi097 to 5722M 2023-07-14T11:57:46.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:45 smithi093 ceph-mon[128922]: osd.8 [v2:172.21.15.97:6816/1965492859,v1:172.21.15.97:6817/1965492859] boot 2023-07-14T11:57:46.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:45 smithi093 ceph-mon[128922]: osdmap e59: 9 total, 9 up, 9 in 2023-07-14T11:57:46.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:45 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T11:57:46.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:45 smithi097 ceph-mon[129313]: purged_snaps scrub starts 2023-07-14T11:57:46.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:45 smithi097 ceph-mon[129313]: purged_snaps scrub ok 2023-07-14T11:57:46.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:45 smithi097 ceph-mon[129313]: Detected new or changed devices on smithi097 2023-07-14T11:57:46.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:45 smithi097 ceph-mon[129313]: Adjusting osd_memory_target on smithi097 to 5722M 2023-07-14T11:57:46.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:45 smithi097 ceph-mon[129313]: osd.8 [v2:172.21.15.97:6816/1965492859,v1:172.21.15.97:6817/1965492859] boot 2023-07-14T11:57:46.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:45 smithi097 ceph-mon[129313]: osdmap e59: 9 total, 9 up, 9 in 2023-07-14T11:57:46.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:45 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T11:57:46.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:45 smithi017 ceph-mon[126339]: purged_snaps scrub starts 2023-07-14T11:57:46.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:45 smithi017 ceph-mon[126339]: purged_snaps scrub ok 2023-07-14T11:57:46.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:45 smithi017 ceph-mon[126339]: Detected new or changed devices on smithi097 2023-07-14T11:57:46.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:45 smithi017 ceph-mon[126339]: Adjusting osd_memory_target on smithi097 to 5722M 2023-07-14T11:57:46.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:45 smithi017 ceph-mon[126339]: osd.8 [v2:172.21.15.97:6816/1965492859,v1:172.21.15.97:6817/1965492859] boot 2023-07-14T11:57:46.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:45 smithi017 ceph-mon[126339]: osdmap e59: 9 total, 9 up, 9 in 2023-07-14T11:57:46.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:45 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T11:57:47.089 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:46 smithi017 ceph-mon[126339]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:57:47.089 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:46 smithi017 ceph-mon[126339]: osdmap e60: 9 total, 9 up, 9 in 2023-07-14T11:57:47.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:46 smithi093 ceph-mon[128922]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:57:47.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:46 smithi093 ceph-mon[128922]: osdmap e60: 9 total, 9 up, 9 in 2023-07-14T11:57:47.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:46 smithi097 ceph-mon[129313]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T11:57:47.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:46 smithi097 ceph-mon[129313]: osdmap e60: 9 total, 9 up, 9 in 2023-07-14T11:57:47.728 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:57:47.746+0000 7f91e0e74700 1 -- 172.21.15.17:0/4292301873 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91dc100d90 msgr2=0x7f91dc1011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:47.728 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:57:47.746+0000 7f91e0e74700 1 --2- 172.21.15.17:0/4292301873 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91dc100d90 0x7f91dc1011e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f91d0009990 tx=0x7f91d0009ca0 comp rx=0 tx=0).stop 2023-07-14T11:57:47.729 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:57:47.746+0000 7f91e0e74700 1 -- 172.21.15.17:0/4292301873 shutdown_connections 2023-07-14T11:57:47.729 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:57:47.746+0000 7f91e0e74700 1 --2- 172.21.15.17:0/4292301873 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f91dc0ffb90 0x7f91dc0fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:47.729 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:57:47.746+0000 7f91e0e74700 1 --2- 172.21.15.17:0/4292301873 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91dc101720 0x7f91dc107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:47.729 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:57:47.746+0000 7f91e0e74700 1 --2- 172.21.15.17:0/4292301873 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91dc100d90 0x7f91dc1011e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:47.729 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:57:47.746+0000 7f91e0e74700 1 -- 172.21.15.17:0/4292301873 >> 172.21.15.17:0/4292301873 conn(0x7f91dc0fb1a0 msgr2=0x7f91dc0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:57:47.729 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:57:47.746+0000 7f91e0e74700 1 -- 172.21.15.17:0/4292301873 shutdown_connections 2023-07-14T11:57:47.729 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:57:47.746+0000 7f91e0e74700 1 -- 172.21.15.17:0/4292301873 wait complete. 2023-07-14T11:57:47.731 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:57:47.747+0000 7f91e0e74700 1 Processor -- start 2023-07-14T11:57:47.732 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:57:47.747+0000 7f91e0e74700 1 -- start start 2023-07-14T11:57:47.732 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:57:47.748+0000 7f91e0e74700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91dc0ffb90 0x7f91dc0742b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:47.732 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:57:47.748+0000 7f91e0e74700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91dc100d90 0x7f91dc072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:47.732 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:57:47.748+0000 7f91e0e74700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f91dc101720 0x7f91dc072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:47.732 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:57:47.748+0000 7f91e0e74700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc0735c0 con 0x7f91dc100d90 2023-07-14T11:57:47.733 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:57:47.748+0000 7f91e0e74700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc073730 con 0x7f91dc101720 2023-07-14T11:57:47.733 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:57:47.748+0000 7f91e0e74700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc19f8e0 con 0x7f91dc0ffb90 2023-07-14T11:57:47.733 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:57:47.748+0000 7f91d9d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91dc100d90 0x7f91dc072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:47.733 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:57:47.748+0000 7f91d9d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91dc100d90 0x7f91dc072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51066/0 (socket says 172.21.15.17:51066) 2023-07-14T11:57:47.733 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:57:47.748+0000 7f91d9d9b700 1 -- 172.21.15.17:0/944234880 learned_addr learned my addr 172.21.15.17:0/944234880 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:57:47.733 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:57:47.748+0000 7f91da59c700 1 --2- 172.21.15.17:0/944234880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91dc0ffb90 0x7f91dc0742b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:47.733 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:57:47.748+0000 7f91dad9d700 1 --2- 172.21.15.17:0/944234880 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f91dc101720 0x7f91dc072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:47.734 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:57:47.748+0000 7f91d9d9b700 1 -- 172.21.15.17:0/944234880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91dc0ffb90 msgr2=0x7f91dc0742b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:47.734 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:57:47.748+0000 7f91d9d9b700 1 --2- 172.21.15.17:0/944234880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91dc0ffb90 0x7f91dc0742b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:47.734 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:57:47.748+0000 7f91d9d9b700 1 -- 172.21.15.17:0/944234880 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f91dc101720 msgr2=0x7f91dc072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:57:47.734 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:57:47.748+0000 7f91d9d9b700 1 --2- 172.21.15.17:0/944234880 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f91dc101720 0x7f91dc072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:57:47.734 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:57:47.748+0000 7f91d9d9b700 1 -- 172.21.15.17:0/944234880 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d00095f0 con 0x7f91dc100d90 2023-07-14T11:57:47.734 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:57:47.749+0000 7f91d9d9b700 1 --2- 172.21.15.17:0/944234880 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91dc100d90 0x7f91dc072900 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f91d0000c00 tx=0x7f91d000f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:47.734 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:57:47.749+0000 7f91cb7fe700 1 -- 172.21.15.17:0/944234880 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d001d070 con 0x7f91dc100d90 2023-07-14T11:57:47.734 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:57:47.749+0000 7f91cb7fe700 1 -- 172.21.15.17:0/944234880 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91d000fb50 con 0x7f91dc100d90 2023-07-14T11:57:47.735 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:57:47.749+0000 7f91e0e74700 1 -- 172.21.15.17:0/944234880 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91dc19fb60 con 0x7f91dc100d90 2023-07-14T11:57:47.735 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:57:47.749+0000 7f91cb7fe700 1 -- 172.21.15.17:0/944234880 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d00173f0 con 0x7f91dc100d90 2023-07-14T11:57:47.735 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:57:47.750+0000 7f91e0e74700 1 -- 172.21.15.17:0/944234880 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91dc1a0050 con 0x7f91dc100d90 2023-07-14T11:57:47.735 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:57:47.751+0000 7f91e0e74700 1 -- 172.21.15.17:0/944234880 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91dc107190 con 0x7f91dc100d90 2023-07-14T11:57:47.736 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:57:47.753+0000 7f91cb7fe700 1 -- 172.21.15.17:0/944234880 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f91d000fcc0 con 0x7f91dc100d90 2023-07-14T11:57:47.736 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:57:47.754+0000 7f91cb7fe700 1 --2- 172.21.15.17:0/944234880 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f91c40a11d0 0x7f91c40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:57:47.736 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:57:47.754+0000 7f91cb7fe700 1 -- 172.21.15.17:0/944234880 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5858+0+0 (secure 0 0 0) 0x7f91d00cd080 con 0x7f91dc100d90 2023-07-14T11:57:47.736 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:57:47.754+0000 7f91da59c700 1 --2- 172.21.15.17:0/944234880 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f91c40a11d0 0x7f91c40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:57:47.737 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:57:47.755+0000 7f91da59c700 1 --2- 172.21.15.17:0/944234880 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f91c40a11d0 0x7f91c40a3680 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f91dc100bf0 tx=0x7f91cc007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:57:47.739 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:57:47.758+0000 7f91cb7fe700 1 -- 172.21.15.17:0/944234880 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f91d0097150 con 0x7f91dc100d90 2023-07-14T11:57:47.956 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:57:47.975+0000 7f91e0e74700 1 -- 172.21.15.17:0/944234880 --> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f91dc060950 con 0x7f91c40a11d0 2023-07-14T11:57:48.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:47 smithi093 ceph-mon[128922]: osdmap e61: 9 total, 9 up, 9 in 2023-07-14T11:57:48.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:47 smithi097 ceph-mon[129313]: osdmap e61: 9 total, 9 up, 9 in 2023-07-14T11:57:48.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:47 smithi017 ceph-mon[126339]: osdmap e61: 9 total, 9 up, 9 in 2023-07-14T11:57:49.039 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:48 smithi017 ceph-mon[126339]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:57:49.039 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:48 smithi017 ceph-mon[126339]: osdmap e62: 9 total, 9 up, 9 in 2023-07-14T11:57:49.039 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:48 smithi017 ceph-mon[126339]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:57:49.040 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:48 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:57:49.040 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:48 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:57:49.040 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:48 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:57:49.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:48 smithi093 ceph-mon[128922]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:57:49.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:48 smithi093 ceph-mon[128922]: osdmap e62: 9 total, 9 up, 9 in 2023-07-14T11:57:49.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:48 smithi093 ceph-mon[128922]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:57:49.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:48 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:57:49.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:48 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:57:49.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:48 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:57:49.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:48 smithi097 ceph-mon[129313]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:57:49.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:48 smithi097 ceph-mon[129313]: osdmap e62: 9 total, 9 up, 9 in 2023-07-14T11:57:49.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:48 smithi097 ceph-mon[129313]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:57:49.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:48 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:57:49.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:48 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:57:49.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:48 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:57:49.997 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:49 smithi017 ceph-mon[126339]: osdmap e63: 9 total, 9 up, 9 in 2023-07-14T11:57:50.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:49 smithi093 ceph-mon[128922]: osdmap e63: 9 total, 9 up, 9 in 2023-07-14T11:57:50.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:49 smithi097 ceph-mon[129313]: osdmap e63: 9 total, 9 up, 9 in 2023-07-14T11:57:51.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:50 smithi093 ceph-mon[128922]: pgmap v195: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-14T11:57:51.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:50 smithi097 ceph-mon[129313]: pgmap v195: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-14T11:57:51.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:50 smithi017 ceph-mon[126339]: pgmap v195: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-14T11:57:52.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:51 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/106521378' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1197b8a3-c09f-4c57-aba1-9e988a1620b4"}]: dispatch 2023-07-14T11:57:52.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:51 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/106521378' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1197b8a3-c09f-4c57-aba1-9e988a1620b4"}]': finished 2023-07-14T11:57:52.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:51 smithi093 ceph-mon[128922]: osdmap e64: 10 total, 9 up, 10 in 2023-07-14T11:57:52.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:51 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T11:57:52.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:51 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/2015480438' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:57:52.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:51 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/106521378' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1197b8a3-c09f-4c57-aba1-9e988a1620b4"}]: dispatch 2023-07-14T11:57:52.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:51 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/106521378' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1197b8a3-c09f-4c57-aba1-9e988a1620b4"}]': finished 2023-07-14T11:57:52.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:51 smithi097 ceph-mon[129313]: osdmap e64: 10 total, 9 up, 10 in 2023-07-14T11:57:52.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:51 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T11:57:52.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:51 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/2015480438' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:57:52.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:51 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/106521378' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1197b8a3-c09f-4c57-aba1-9e988a1620b4"}]: dispatch 2023-07-14T11:57:52.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:51 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/106521378' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1197b8a3-c09f-4c57-aba1-9e988a1620b4"}]': finished 2023-07-14T11:57:52.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:51 smithi017 ceph-mon[126339]: osdmap e64: 10 total, 9 up, 10 in 2023-07-14T11:57:52.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:51 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T11:57:52.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:51 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/2015480438' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:57:53.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:53 smithi093 ceph-mon[128922]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 117 KiB/s, 0 objects/s recovering 2023-07-14T11:57:53.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:53 smithi097 ceph-mon[129313]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 117 KiB/s, 0 objects/s recovering 2023-07-14T11:57:53.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:53 smithi017 ceph-mon[126339]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 117 KiB/s, 0 objects/s recovering 2023-07-14T11:57:54.861 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:54 smithi093 ceph-mon[128922]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-14T11:57:54.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:54 smithi097 ceph-mon[129313]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-14T11:57:54.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:54 smithi017 ceph-mon[126339]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-14T11:57:56.800 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:56 smithi017 ceph-mon[126339]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2023-07-14T11:57:57.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:56 smithi093 ceph-mon[128922]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2023-07-14T11:57:57.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:56 smithi097 ceph-mon[129313]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2023-07-14T11:57:58.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:58 smithi017 ceph-mon[126339]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:57:58.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:58 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-07-14T11:57:58.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:58 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:57:59.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:58 smithi093 ceph-mon[128922]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:57:59.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:58 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-07-14T11:57:59.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:58 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:57:59.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:58 smithi097 ceph-mon[129313]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:57:59.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:58 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-07-14T11:57:59.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:58 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:57:59.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:57:59 smithi017 ceph-mon[126339]: Deploying daemon osd.9 on smithi017 2023-07-14T11:58:00.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:57:59 smithi093 ceph-mon[128922]: Deploying daemon osd.9 on smithi017 2023-07-14T11:58:00.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:57:59 smithi097 ceph-mon[129313]: Deploying daemon osd.9 on smithi017 2023-07-14T11:58:01.104 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:00 smithi017 ceph-mon[126339]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:58:01.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:00 smithi093 ceph-mon[128922]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:58:01.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:00 smithi097 ceph-mon[129313]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:58:03.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:02 smithi093 ceph-mon[128922]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:58:03.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:02 smithi097 ceph-mon[129313]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:58:03.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:02 smithi017 ceph-mon[126339]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:58:04.704 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:04 smithi017 ceph-mon[126339]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:58:04.861 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:04 smithi093 ceph-mon[128922]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:58:04.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:04 smithi097 ceph-mon[129313]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:58:06.309 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:06 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:06.309 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:06 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:58:06.309 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:06 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:06.309 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:06 smithi017 ceph-mon[126339]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:58:06.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:06 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:06.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:06 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:58:06.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:06 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:06.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:06 smithi093 ceph-mon[128922]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:58:06.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:06 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:06.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:06 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:58:06.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:06 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:06.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:06 smithi097 ceph-mon[129313]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:58:08.886 INFO:teuthology.orchestra.run.smithi017.stdout:Created osd(s) 9 on host 'smithi017' 2023-07-14T11:58:08.887 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:08.904+0000 7f91cb7fe700 1 -- 172.21.15.17:0/944234880 <== mgr.14152 v2:172.21.15.17:6800/4237449858 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f91dc060950 con 0x7f91c40a11d0 2023-07-14T11:58:08.888 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:08.907+0000 7f91e0e74700 1 -- 172.21.15.17:0/944234880 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f91c40a11d0 msgr2=0x7f91c40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:58:08.889 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:08.907+0000 7f91e0e74700 1 --2- 172.21.15.17:0/944234880 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f91c40a11d0 0x7f91c40a3680 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f91dc100bf0 tx=0x7f91cc007400 comp rx=0 tx=0).stop 2023-07-14T11:58:08.889 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:08.907+0000 7f91e0e74700 1 -- 172.21.15.17:0/944234880 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91dc100d90 msgr2=0x7f91dc072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:58:08.889 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:08.907+0000 7f91e0e74700 1 --2- 172.21.15.17:0/944234880 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91dc100d90 0x7f91dc072900 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f91d0000c00 tx=0x7f91d000f710 comp rx=0 tx=0).stop 2023-07-14T11:58:08.889 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:08.907+0000 7f91e0e74700 1 -- 172.21.15.17:0/944234880 shutdown_connections 2023-07-14T11:58:08.889 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:08.907+0000 7f91e0e74700 1 --2- 172.21.15.17:0/944234880 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f91c40a11d0 0x7f91c40a3680 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:58:08.889 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:08.907+0000 7f91e0e74700 1 --2- 172.21.15.17:0/944234880 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f91dc101720 0x7f91dc072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:58:08.889 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:08.907+0000 7f91e0e74700 1 --2- 172.21.15.17:0/944234880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91dc0ffb90 0x7f91dc0742b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:58:08.890 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:08.907+0000 7f91e0e74700 1 --2- 172.21.15.17:0/944234880 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91dc100d90 0x7f91dc072900 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:58:08.890 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:08.907+0000 7f91e0e74700 1 -- 172.21.15.17:0/944234880 >> 172.21.15.17:0/944234880 conn(0x7f91dc0fb1a0 msgr2=0x7f91dc1042d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:58:08.890 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:08.907+0000 7f91e0e74700 1 -- 172.21.15.17:0/944234880 shutdown_connections 2023-07-14T11:58:08.890 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:08.907+0000 7f91e0e74700 1 -- 172.21.15.17:0/944234880 wait complete. 2023-07-14T11:58:09.062 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:08 smithi017 ceph-mon[126339]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:58:09.062 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:08 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:09.063 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:08 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:09.063 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:08 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:58:09.063 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:08 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:58:09.063 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:08 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:09.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:08 smithi093 ceph-mon[128922]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:58:09.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:08 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:09.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:08 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:09.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:08 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:58:09.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:08 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:58:09.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:08 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:09.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:08 smithi097 ceph-mon[129313]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:58:09.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:08 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:09.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:08 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:09.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:08 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:58:09.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:08 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:58:09.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:08 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:09.434 DEBUG:teuthology.orchestra.run.smithi017:osd.9> sudo journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@osd.9.service 2023-07-14T11:58:09.439 INFO:tasks.cephadm:Deploying osd.10 on smithi093 with /dev/vg_nvme/lv_1... 2023-07-14T11:58:09.439 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-07-14T11:58:09.519 INFO:journalctl@ceph.osd.9.smithi017.stdout:-- Logs begin at Fri 2023-07-14 11:32:03 UTC. -- 2023-07-14T11:58:10.004 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.b/config 2023-07-14T11:58:10.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:09 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:10.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:09 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:10.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:09 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:58:10.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:09 smithi093 ceph-mon[128922]: from='osd.9 [v2:172.21.15.17:6826/2512645700,v1:172.21.15.17:6827/2512645700]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-14T11:58:10.127 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:09 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:10.127 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:09 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:10.127 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:09 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:58:10.127 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:09 smithi017 ceph-mon[126339]: from='osd.9 [v2:172.21.15.17:6826/2512645700,v1:172.21.15.17:6827/2512645700]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-14T11:58:10.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:09 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:10.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:09 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:10.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:09 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:58:10.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:09 smithi097 ceph-mon[129313]: from='osd.9 [v2:172.21.15.17:6826/2512645700,v1:172.21.15.17:6827/2512645700]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-14T11:58:11.121 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:10 smithi017 ceph-mon[126339]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:58:11.121 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:10 smithi017 ceph-mon[126339]: from='osd.9 [v2:172.21.15.17:6826/2512645700,v1:172.21.15.17:6827/2512645700]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-07-14T11:58:11.121 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:10 smithi017 ceph-mon[126339]: osdmap e65: 10 total, 9 up, 10 in 2023-07-14T11:58:11.121 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:10 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T11:58:11.121 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:10 smithi017 ceph-mon[126339]: from='osd.9 [v2:172.21.15.17:6826/2512645700,v1:172.21.15.17:6827/2512645700]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2023-07-14T11:58:11.122 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 11:58:10 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T11:58:10.890+0000 7fbdd95fd700 -1 osd.9 0 waiting for initial osdmap 2023-07-14T11:58:11.122 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 11:58:10 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T11:58:10.905+0000 7fbdd53f2700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T11:58:11.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:10 smithi097 ceph-mon[129313]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:58:11.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:10 smithi097 ceph-mon[129313]: from='osd.9 [v2:172.21.15.17:6826/2512645700,v1:172.21.15.17:6827/2512645700]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-07-14T11:58:11.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:10 smithi097 ceph-mon[129313]: osdmap e65: 10 total, 9 up, 10 in 2023-07-14T11:58:11.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:10 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T11:58:11.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:10 smithi097 ceph-mon[129313]: from='osd.9 [v2:172.21.15.17:6826/2512645700,v1:172.21.15.17:6827/2512645700]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2023-07-14T11:58:11.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:10 smithi093 ceph-mon[128922]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:58:11.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:10 smithi093 ceph-mon[128922]: from='osd.9 [v2:172.21.15.17:6826/2512645700,v1:172.21.15.17:6827/2512645700]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-07-14T11:58:11.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:10 smithi093 ceph-mon[128922]: osdmap e65: 10 total, 9 up, 10 in 2023-07-14T11:58:11.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:10 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T11:58:11.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:10 smithi093 ceph-mon[128922]: from='osd.9 [v2:172.21.15.17:6826/2512645700,v1:172.21.15.17:6827/2512645700]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2023-07-14T11:58:11.464 INFO:teuthology.orchestra.run.smithi093.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-07-14T11:58:11.464 INFO:teuthology.orchestra.run.smithi093.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-07-14T11:58:11.464 INFO:teuthology.orchestra.run.smithi093.stderr: stderr: 10+0 records in 2023-07-14T11:58:11.464 INFO:teuthology.orchestra.run.smithi093.stderr:10+0 records out 2023-07-14T11:58:11.465 INFO:teuthology.orchestra.run.smithi093.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111514 s, 940 MB/s 2023-07-14T11:58:11.465 INFO:teuthology.orchestra.run.smithi093.stderr:--> Zapping successful for: 2023-07-14T11:58:11.844 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi093:vg_nvme/lv_1 2023-07-14T11:58:12.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:11 smithi093 ceph-mon[128922]: from='osd.9 [v2:172.21.15.17:6826/2512645700,v1:172.21.15.17:6827/2512645700]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2023-07-14T11:58:12.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:11 smithi093 ceph-mon[128922]: osdmap e66: 10 total, 9 up, 10 in 2023-07-14T11:58:12.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:11 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T11:58:12.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:11 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T11:58:12.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:11 smithi097 ceph-mon[129313]: from='osd.9 [v2:172.21.15.17:6826/2512645700,v1:172.21.15.17:6827/2512645700]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2023-07-14T11:58:12.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:11 smithi097 ceph-mon[129313]: osdmap e66: 10 total, 9 up, 10 in 2023-07-14T11:58:12.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:11 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T11:58:12.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:11 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T11:58:12.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:11 smithi017 ceph-mon[126339]: from='osd.9 [v2:172.21.15.17:6826/2512645700,v1:172.21.15.17:6827/2512645700]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2023-07-14T11:58:12.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:11 smithi017 ceph-mon[126339]: osdmap e66: 10 total, 9 up, 10 in 2023-07-14T11:58:12.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:11 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T11:58:12.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:11 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T11:58:12.396 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.b/config 2023-07-14T11:58:13.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:12 smithi017 ceph-mon[126339]: purged_snaps scrub starts 2023-07-14T11:58:13.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:12 smithi017 ceph-mon[126339]: purged_snaps scrub ok 2023-07-14T11:58:13.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:12 smithi017 ceph-mon[126339]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:58:13.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:12 smithi017 ceph-mon[126339]: osd.9 [v2:172.21.15.17:6826/2512645700,v1:172.21.15.17:6827/2512645700] boot 2023-07-14T11:58:13.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:12 smithi017 ceph-mon[126339]: osdmap e67: 10 total, 10 up, 10 in 2023-07-14T11:58:13.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:12 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T11:58:13.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:12 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:13.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:12 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:13.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:12 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:58:13.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:12 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:58:13.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:12 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:58:13.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:12 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:58:13.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:12 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:13.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:12 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:58:13.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:12 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:58:13.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:12 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:13.230 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:12 smithi093 ceph-mon[128922]: purged_snaps scrub starts 2023-07-14T11:58:13.231 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:12 smithi093 ceph-mon[128922]: purged_snaps scrub ok 2023-07-14T11:58:13.231 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:12 smithi093 ceph-mon[128922]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:58:13.231 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:12 smithi093 ceph-mon[128922]: osd.9 [v2:172.21.15.17:6826/2512645700,v1:172.21.15.17:6827/2512645700] boot 2023-07-14T11:58:13.231 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:12 smithi093 ceph-mon[128922]: osdmap e67: 10 total, 10 up, 10 in 2023-07-14T11:58:13.231 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:12 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T11:58:13.231 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:12 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:13.232 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:12 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:13.232 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:12 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:58:13.232 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:12 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:58:13.232 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:12 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:58:13.232 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:12 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:58:13.232 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:12 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:13.232 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:12 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:58:13.232 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:12 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:58:13.233 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:12 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:13.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:12 smithi097 ceph-mon[129313]: purged_snaps scrub starts 2023-07-14T11:58:13.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:12 smithi097 ceph-mon[129313]: purged_snaps scrub ok 2023-07-14T11:58:13.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:12 smithi097 ceph-mon[129313]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T11:58:13.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:12 smithi097 ceph-mon[129313]: osd.9 [v2:172.21.15.17:6826/2512645700,v1:172.21.15.17:6827/2512645700] boot 2023-07-14T11:58:13.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:12 smithi097 ceph-mon[129313]: osdmap e67: 10 total, 10 up, 10 in 2023-07-14T11:58:13.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:12 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T11:58:13.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:12 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:13.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:12 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:13.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:12 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:58:13.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:12 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:58:13.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:12 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:58:13.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:12 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:58:13.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:12 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:13.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:12 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:58:13.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:12 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:58:13.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:12 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:13.678 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:13.694+0000 7f312194b700 1 -- 172.21.15.93:0/549195897 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f311c100e20 msgr2=0x7f311c101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:58:13.679 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:13.694+0000 7f312194b700 1 --2- 172.21.15.93:0/549195897 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f311c100e20 0x7f311c101270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f310c009990 tx=0x7f310c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:58:13.679 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:13.694+0000 7f312194b700 1 -- 172.21.15.93:0/549195897 shutdown_connections 2023-07-14T11:58:13.679 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:13.694+0000 7f312194b700 1 --2- 172.21.15.93:0/549195897 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f311c1017b0 0x7f311c108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:58:13.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:13.695+0000 7f312194b700 1 --2- 172.21.15.93:0/549195897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f311c0ffc20 0x7f311c100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:58:13.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:13.695+0000 7f312194b700 1 --2- 172.21.15.93:0/549195897 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f311c100e20 0x7f311c101270 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:58:13.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:13.695+0000 7f312194b700 1 -- 172.21.15.93:0/549195897 >> 172.21.15.93:0/549195897 conn(0x7f311c0fb250 msgr2=0x7f311c0fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:58:13.681 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:13.695+0000 7f312194b700 1 -- 172.21.15.93:0/549195897 shutdown_connections 2023-07-14T11:58:13.681 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:13.695+0000 7f312194b700 1 -- 172.21.15.93:0/549195897 wait complete. 2023-07-14T11:58:13.681 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:13.695+0000 7f312194b700 1 Processor -- start 2023-07-14T11:58:13.681 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:13.695+0000 7f312194b700 1 -- start start 2023-07-14T11:58:13.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:13.695+0000 7f312194b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f311c0ffc20 0x7f311c1958e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:58:13.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:13.695+0000 7f312194b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f311c100e20 0x7f311c195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:58:13.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:13.696+0000 7f312194b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f311c1017b0 0x7f311c19cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:58:13.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:13.696+0000 7f312194b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311c19d470 con 0x7f311c0ffc20 2023-07-14T11:58:13.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:13.696+0000 7f312194b700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311c19d5b0 con 0x7f311c100e20 2023-07-14T11:58:13.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:13.696+0000 7f312194b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311c19d740 con 0x7f311c1017b0 2023-07-14T11:58:13.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:13.696+0000 7f311a7fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f311c100e20 0x7f311c195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:58:13.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:13.696+0000 7f311a7fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f311c100e20 0x7f311c195e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:57930/0 (socket says 172.21.15.93:57930) 2023-07-14T11:58:13.684 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:13.696+0000 7f311a7fc700 1 -- 172.21.15.93:0/1885503737 learned_addr learned my addr 172.21.15.93:0/1885503737 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-14T11:58:13.684 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:13.696+0000 7f311affd700 1 --2- 172.21.15.93:0/1885503737 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f311c0ffc20 0x7f311c1958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:58:13.684 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:13.696+0000 7f311b7fe700 1 --2- 172.21.15.93:0/1885503737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f311c1017b0 0x7f311c19cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:58:13.684 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:13.696+0000 7f311a7fc700 1 -- 172.21.15.93:0/1885503737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f311c1017b0 msgr2=0x7f311c19cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:58:13.684 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:13.696+0000 7f311a7fc700 1 --2- 172.21.15.93:0/1885503737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f311c1017b0 0x7f311c19cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:58:13.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:13.696+0000 7f311a7fc700 1 -- 172.21.15.93:0/1885503737 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f311c0ffc20 msgr2=0x7f311c1958e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:58:13.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:13.696+0000 7f311a7fc700 1 --2- 172.21.15.93:0/1885503737 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f311c0ffc20 0x7f311c1958e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:58:13.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:13.696+0000 7f311a7fc700 1 -- 172.21.15.93:0/1885503737 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f310c0095f0 con 0x7f311c100e20 2023-07-14T11:58:13.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:13.696+0000 7f311affd700 1 --2- 172.21.15.93:0/1885503737 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f311c0ffc20 0x7f311c1958e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:58:13.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:13.696+0000 7f311a7fc700 1 --2- 172.21.15.93:0/1885503737 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f311c100e20 0x7f311c195e20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f310c0059d0 tx=0x7f310c00f510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:58:13.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:13.696+0000 7f3120949700 1 -- 172.21.15.93:0/1885503737 <== mon.2 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310c01d070 con 0x7f311c100e20 2023-07-14T11:58:13.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:13.696+0000 7f3120949700 1 -- 172.21.15.93:0/1885503737 <== mon.2 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f310c00fd60 con 0x7f311c100e20 2023-07-14T11:58:13.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:13.696+0000 7f3120949700 1 -- 172.21.15.93:0/1885503737 <== mon.2 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310c017650 con 0x7f311c100e20 2023-07-14T11:58:13.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:13.696+0000 7f312194b700 1 -- 172.21.15.93:0/1885503737 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f311c19d9c0 con 0x7f311c100e20 2023-07-14T11:58:13.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:13.696+0000 7f312194b700 1 -- 172.21.15.93:0/1885503737 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f311c19dd80 con 0x7f311c100e20 2023-07-14T11:58:13.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:13.697+0000 7f312194b700 1 -- 172.21.15.93:0/1885503737 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f311c107250 con 0x7f311c100e20 2023-07-14T11:58:13.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:13.699+0000 7f3120949700 1 -- 172.21.15.93:0/1885503737 <== mon.2 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f310c004830 con 0x7f311c100e20 2023-07-14T11:58:13.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:13.699+0000 7f3120949700 1 --2- 172.21.15.93:0/1885503737 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f31080a1150 0x7f31080a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:58:13.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:13.699+0000 7f3120949700 1 -- 172.21.15.93:0/1885503737 <== mon.2 v2:172.21.15.93:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f310c0cd8b0 con 0x7f311c100e20 2023-07-14T11:58:13.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:13.700+0000 7f311affd700 1 --2- 172.21.15.93:0/1885503737 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f31080a1150 0x7f31080a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:58:13.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:13.700+0000 7f311affd700 1 --2- 172.21.15.93:0/1885503737 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f31080a1150 0x7f31080a3600 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3104005fd0 tx=0x7f3104005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:58:13.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:13.702+0000 7f3120949700 1 -- 172.21.15.93:0/1885503737 <== mon.2 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f310c097890 con 0x7f311c100e20 2023-07-14T11:58:13.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:13.895+0000 7f312194b700 1 -- 172.21.15.93:0/1885503737 --> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f311c060950 con 0x7f31080a1150 2023-07-14T11:58:14.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:13 smithi017 ceph-mon[126339]: Detected new or changed devices on smithi017 2023-07-14T11:58:14.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:13 smithi017 ceph-mon[126339]: Adjusting osd_memory_target on smithi017 to 4318M 2023-07-14T11:58:14.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:13 smithi017 ceph-mon[126339]: osdmap e68: 10 total, 10 up, 10 in 2023-07-14T11:58:14.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:13 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:58:14.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:13 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:58:14.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:13 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:58:14.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:13 smithi093 ceph-mon[128922]: Detected new or changed devices on smithi017 2023-07-14T11:58:14.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:13 smithi093 ceph-mon[128922]: Adjusting osd_memory_target on smithi017 to 4318M 2023-07-14T11:58:14.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:13 smithi093 ceph-mon[128922]: osdmap e68: 10 total, 10 up, 10 in 2023-07-14T11:58:14.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:13 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:58:14.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:13 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:58:14.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:13 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:58:14.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:13 smithi097 ceph-mon[129313]: Detected new or changed devices on smithi017 2023-07-14T11:58:14.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:13 smithi097 ceph-mon[129313]: Adjusting osd_memory_target on smithi017 to 4318M 2023-07-14T11:58:14.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:13 smithi097 ceph-mon[129313]: osdmap e68: 10 total, 10 up, 10 in 2023-07-14T11:58:14.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:13 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:58:14.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:13 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:58:14.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:13 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:58:14.971 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:14 smithi093 ceph-mon[128922]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:58:14.971 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:14 smithi093 ceph-mon[128922]: from='client.24356 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:58:15.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:14 smithi017 ceph-mon[126339]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:58:15.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:14 smithi017 ceph-mon[126339]: from='client.24356 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:58:15.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:14 smithi097 ceph-mon[129313]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:58:15.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:14 smithi097 ceph-mon[129313]: from='client.24356 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:58:17.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:16 smithi093 ceph-mon[128922]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:58:17.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:16 smithi093 ceph-mon[128922]: from='client.? 172.21.15.93:0/3381372012' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f99918c2-cdd8-4a7a-be2e-d1d6e0f828a4"}]: dispatch 2023-07-14T11:58:17.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:16 smithi093 ceph-mon[128922]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f99918c2-cdd8-4a7a-be2e-d1d6e0f828a4"}]: dispatch 2023-07-14T11:58:17.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:16 smithi093 ceph-mon[128922]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f99918c2-cdd8-4a7a-be2e-d1d6e0f828a4"}]': finished 2023-07-14T11:58:17.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:16 smithi093 ceph-mon[128922]: osdmap e69: 11 total, 10 up, 11 in 2023-07-14T11:58:17.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:16 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T11:58:17.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:16 smithi093 ceph-mon[128922]: from='client.? 172.21.15.93:0/2460136858' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:58:17.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:16 smithi097 ceph-mon[129313]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:58:17.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:16 smithi097 ceph-mon[129313]: from='client.? 172.21.15.93:0/3381372012' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f99918c2-cdd8-4a7a-be2e-d1d6e0f828a4"}]: dispatch 2023-07-14T11:58:17.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:16 smithi097 ceph-mon[129313]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f99918c2-cdd8-4a7a-be2e-d1d6e0f828a4"}]: dispatch 2023-07-14T11:58:17.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:16 smithi097 ceph-mon[129313]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f99918c2-cdd8-4a7a-be2e-d1d6e0f828a4"}]': finished 2023-07-14T11:58:17.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:16 smithi097 ceph-mon[129313]: osdmap e69: 11 total, 10 up, 11 in 2023-07-14T11:58:17.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:16 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T11:58:17.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:16 smithi097 ceph-mon[129313]: from='client.? 172.21.15.93:0/2460136858' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:58:17.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:16 smithi017 ceph-mon[126339]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:58:17.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:16 smithi017 ceph-mon[126339]: from='client.? 172.21.15.93:0/3381372012' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f99918c2-cdd8-4a7a-be2e-d1d6e0f828a4"}]: dispatch 2023-07-14T11:58:17.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:16 smithi017 ceph-mon[126339]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f99918c2-cdd8-4a7a-be2e-d1d6e0f828a4"}]: dispatch 2023-07-14T11:58:17.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:16 smithi017 ceph-mon[126339]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f99918c2-cdd8-4a7a-be2e-d1d6e0f828a4"}]': finished 2023-07-14T11:58:17.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:16 smithi017 ceph-mon[126339]: osdmap e69: 11 total, 10 up, 11 in 2023-07-14T11:58:17.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:16 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T11:58:17.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:16 smithi017 ceph-mon[126339]: from='client.? 172.21.15.93:0/2460136858' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:58:19.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:18 smithi093 ceph-mon[128922]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:58:19.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:18 smithi097 ceph-mon[129313]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:58:19.394 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:18 smithi017 ceph-mon[126339]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:58:21.199 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:20 smithi093 ceph-mon[128922]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:58:21.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:20 smithi097 ceph-mon[129313]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:58:21.394 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:20 smithi017 ceph-mon[126339]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:58:23.168 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:22 smithi093 ceph-mon[128922]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:58:23.169 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:22 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-07-14T11:58:23.169 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:22 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:58:23.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:22 smithi097 ceph-mon[129313]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:58:23.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:22 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-07-14T11:58:23.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:22 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:58:23.394 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:22 smithi017 ceph-mon[126339]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:58:23.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:22 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-07-14T11:58:23.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:22 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:58:24.060 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:23 smithi093 ceph-mon[128922]: Deploying daemon osd.10 on smithi093 2023-07-14T11:58:24.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:23 smithi097 ceph-mon[129313]: Deploying daemon osd.10 on smithi093 2023-07-14T11:58:24.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:23 smithi017 ceph-mon[126339]: Deploying daemon osd.10 on smithi093 2023-07-14T11:58:25.028 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:24 smithi093 ceph-mon[128922]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:58:25.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:24 smithi097 ceph-mon[129313]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:58:25.394 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:24 smithi017 ceph-mon[126339]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:58:27.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:26 smithi093 ceph-mon[128922]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:58:27.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:26 smithi097 ceph-mon[129313]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:58:27.394 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:26 smithi017 ceph-mon[126339]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:58:28.103 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:27 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:28.104 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:27 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:28.104 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:27 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:58:28.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:27 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:28.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:27 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:28.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:27 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:58:28.394 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:27 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:28.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:27 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:28.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:27 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:58:29.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:28 smithi093 ceph-mon[128922]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:58:29.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:28 smithi097 ceph-mon[129313]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:58:29.394 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:28 smithi017 ceph-mon[126339]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:58:29.792 INFO:teuthology.orchestra.run.smithi093.stdout:Created osd(s) 10 on host 'smithi093' 2023-07-14T11:58:29.793 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:29.810+0000 7f3120949700 1 -- 172.21.15.93:0/1885503737 <== mgr.14152 v2:172.21.15.17:6800/4237449858 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f311c060950 con 0x7f31080a1150 2023-07-14T11:58:29.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:29.813+0000 7f312194b700 1 -- 172.21.15.93:0/1885503737 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f31080a1150 msgr2=0x7f31080a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:58:29.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:29.813+0000 7f312194b700 1 --2- 172.21.15.93:0/1885503737 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f31080a1150 0x7f31080a3600 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3104005fd0 tx=0x7f3104005f00 comp rx=0 tx=0).stop 2023-07-14T11:58:29.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:29.813+0000 7f312194b700 1 -- 172.21.15.93:0/1885503737 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f311c100e20 msgr2=0x7f311c195e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:58:29.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:29.813+0000 7f312194b700 1 --2- 172.21.15.93:0/1885503737 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f311c100e20 0x7f311c195e20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f310c0059d0 tx=0x7f310c00f510 comp rx=0 tx=0).stop 2023-07-14T11:58:29.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:29.813+0000 7f312194b700 1 -- 172.21.15.93:0/1885503737 shutdown_connections 2023-07-14T11:58:29.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:29.813+0000 7f312194b700 1 --2- 172.21.15.93:0/1885503737 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f31080a1150 0x7f31080a3600 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:58:29.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:29.813+0000 7f312194b700 1 --2- 172.21.15.93:0/1885503737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f311c1017b0 0x7f311c19cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:58:29.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:29.813+0000 7f312194b700 1 --2- 172.21.15.93:0/1885503737 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f311c0ffc20 0x7f311c1958e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:58:29.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:29.813+0000 7f312194b700 1 --2- 172.21.15.93:0/1885503737 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f311c100e20 0x7f311c195e20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:58:29.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:29.813+0000 7f312194b700 1 -- 172.21.15.93:0/1885503737 >> 172.21.15.93:0/1885503737 conn(0x7f311c0fb250 msgr2=0x7f311c104390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:58:29.798 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:29.813+0000 7f312194b700 1 -- 172.21.15.93:0/1885503737 shutdown_connections 2023-07-14T11:58:29.798 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-14T11:58:29.813+0000 7f312194b700 1 -- 172.21.15.93:0/1885503737 wait complete. 2023-07-14T11:58:30.191 DEBUG:teuthology.orchestra.run.smithi093:osd.10> sudo journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@osd.10.service 2023-07-14T11:58:30.194 INFO:tasks.cephadm:Deploying osd.11 on smithi097 with /dev/vg_nvme/lv_1... 2023-07-14T11:58:30.194 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-07-14T11:58:30.233 INFO:journalctl@ceph.osd.10.smithi093.stdout:-- Logs begin at Fri 2023-07-14 11:24:16 UTC. -- 2023-07-14T11:58:30.759 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T11:58:30.861 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:30 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:30.862 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:30 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:30.862 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:30 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:58:30.862 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:30 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:58:30.862 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:30 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:30.862 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:30 smithi093 ceph-mon[128922]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:58:30.862 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:30 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:30.862 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:30 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:30.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:30 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:58:30.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:30 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:30.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:30 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:30.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:30 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:58:30.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:30 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:58:30.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:30 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:30.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:30 smithi097 ceph-mon[129313]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:58:30.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:30 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:30.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:30 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:30.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:30 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:58:30.894 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:30 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:30.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:30 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:30.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:30 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:58:30.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:30 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:58:30.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:30 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:30.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:30 smithi017 ceph-mon[126339]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:58:30.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:30 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:30.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:30 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:30.896 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:30 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:58:31.260 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 11:58:30 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T11:58:30.936+0000 7f879c585640 -1 osd.10 0 log_to_monitors true 2023-07-14T11:58:31.862 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:31 smithi093 ceph-mon[128922]: from='osd.10 [v2:172.21.15.93:6824/264429558,v1:172.21.15.93:6825/264429558]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-14T11:58:31.862 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:31 smithi093 ceph-mon[128922]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-14T11:58:31.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:31 smithi097 ceph-mon[129313]: from='osd.10 [v2:172.21.15.93:6824/264429558,v1:172.21.15.93:6825/264429558]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-14T11:58:31.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:31 smithi097 ceph-mon[129313]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-14T11:58:31.894 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:31 smithi017 ceph-mon[126339]: from='osd.10 [v2:172.21.15.93:6824/264429558,v1:172.21.15.93:6825/264429558]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-14T11:58:31.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:31 smithi017 ceph-mon[126339]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-14T11:58:32.264 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-07-14T11:58:32.264 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-07-14T11:58:32.264 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2023-07-14T11:58:32.264 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2023-07-14T11:58:32.265 INFO:teuthology.orchestra.run.smithi097.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0130668 s, 802 MB/s 2023-07-14T11:58:32.265 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2023-07-14T11:58:32.699 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi097:vg_nvme/lv_1 2023-07-14T11:58:32.861 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 11:58:32 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T11:58:32.641+0000 7f8791bfd700 -1 osd.10 0 waiting for initial osdmap 2023-07-14T11:58:32.861 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 11:58:32 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T11:58:32.657+0000 7f878d9f2700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T11:58:32.862 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:32 smithi093 ceph-mon[128922]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-07-14T11:58:32.862 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:32 smithi093 ceph-mon[128922]: osdmap e70: 11 total, 10 up, 11 in 2023-07-14T11:58:32.862 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:32 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T11:58:32.862 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:32 smithi093 ceph-mon[128922]: from='osd.10 [v2:172.21.15.93:6824/264429558,v1:172.21.15.93:6825/264429558]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-07-14T11:58:32.862 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:32 smithi093 ceph-mon[128922]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-07-14T11:58:32.862 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:32 smithi093 ceph-mon[128922]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:58:32.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:32 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:32.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:32 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:32.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:32 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:58:32.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:32 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:58:32.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:32 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:58:32.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:32 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:58:32.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:32 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:32.864 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:32 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:58:32.864 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:32 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:58:32.864 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:32 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:32.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:32 smithi097 ceph-mon[129313]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-07-14T11:58:32.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:32 smithi097 ceph-mon[129313]: osdmap e70: 11 total, 10 up, 11 in 2023-07-14T11:58:32.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:32 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T11:58:32.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:32 smithi097 ceph-mon[129313]: from='osd.10 [v2:172.21.15.93:6824/264429558,v1:172.21.15.93:6825/264429558]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-07-14T11:58:32.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:32 smithi097 ceph-mon[129313]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-07-14T11:58:32.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:32 smithi097 ceph-mon[129313]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:58:32.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:32 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:32.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:32 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:32.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:32 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:58:32.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:32 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:58:32.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:32 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:58:32.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:32 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:58:32.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:32 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:32.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:32 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:58:32.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:32 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:58:32.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:32 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:32.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:32 smithi017 ceph-mon[126339]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-07-14T11:58:32.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:32 smithi017 ceph-mon[126339]: osdmap e70: 11 total, 10 up, 11 in 2023-07-14T11:58:32.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:32 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T11:58:32.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:32 smithi017 ceph-mon[126339]: from='osd.10 [v2:172.21.15.93:6824/264429558,v1:172.21.15.93:6825/264429558]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-07-14T11:58:32.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:32 smithi017 ceph-mon[126339]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-07-14T11:58:32.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:32 smithi017 ceph-mon[126339]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T11:58:32.896 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:32 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:32.896 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:32 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:32.896 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:32 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:58:32.896 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:32 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:58:32.896 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:32 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:58:32.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:32 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:58:32.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:32 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:32.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:32 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:58:32.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:32 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:58:32.897 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:32 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:33.252 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T11:58:33.718 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:33 smithi097 ceph-mon[129313]: Detected new or changed devices on smithi093 2023-07-14T11:58:33.718 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:33 smithi097 ceph-mon[129313]: Adjusting osd_memory_target on smithi093 to 4292M 2023-07-14T11:58:33.718 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:33 smithi097 ceph-mon[129313]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-07-14T11:58:33.718 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:33 smithi097 ceph-mon[129313]: osdmap e71: 11 total, 10 up, 11 in 2023-07-14T11:58:33.718 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:33 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T11:58:33.718 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:33 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T11:58:33.719 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:33 smithi097 ceph-mon[129313]: osd.10 [v2:172.21.15.93:6824/264429558,v1:172.21.15.93:6825/264429558] boot 2023-07-14T11:58:33.719 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:33 smithi097 ceph-mon[129313]: osdmap e72: 11 total, 11 up, 11 in 2023-07-14T11:58:33.719 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:33 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T11:58:33.894 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:33 smithi017 ceph-mon[126339]: Detected new or changed devices on smithi093 2023-07-14T11:58:33.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:33 smithi017 ceph-mon[126339]: Adjusting osd_memory_target on smithi093 to 4292M 2023-07-14T11:58:33.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:33 smithi017 ceph-mon[126339]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-07-14T11:58:33.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:33 smithi017 ceph-mon[126339]: osdmap e71: 11 total, 10 up, 11 in 2023-07-14T11:58:33.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:33 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T11:58:33.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:33 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T11:58:33.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:33 smithi017 ceph-mon[126339]: osd.10 [v2:172.21.15.93:6824/264429558,v1:172.21.15.93:6825/264429558] boot 2023-07-14T11:58:33.896 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:33 smithi017 ceph-mon[126339]: osdmap e72: 11 total, 11 up, 11 in 2023-07-14T11:58:33.896 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:33 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T11:58:34.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:33 smithi093 ceph-mon[128922]: Detected new or changed devices on smithi093 2023-07-14T11:58:34.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:33 smithi093 ceph-mon[128922]: Adjusting osd_memory_target on smithi093 to 4292M 2023-07-14T11:58:34.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:33 smithi093 ceph-mon[128922]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-07-14T11:58:34.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:33 smithi093 ceph-mon[128922]: osdmap e71: 11 total, 10 up, 11 in 2023-07-14T11:58:34.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:33 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T11:58:34.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:33 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T11:58:34.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:33 smithi093 ceph-mon[128922]: osd.10 [v2:172.21.15.93:6824/264429558,v1:172.21.15.93:6825/264429558] boot 2023-07-14T11:58:34.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:33 smithi093 ceph-mon[128922]: osdmap e72: 11 total, 11 up, 11 in 2023-07-14T11:58:34.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:33 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T11:58:34.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:34.502+0000 7f7840443700 1 -- 172.21.15.97:0/665427921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7838100610 msgr2=0x7f7838106ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:58:34.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:34.502+0000 7f7840443700 1 --2- 172.21.15.97:0/665427921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7838100610 0x7f7838106ed0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f783000b0a0 tx=0x7f783000b3b0 comp rx=0 tx=0).stop 2023-07-14T11:58:34.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:34.502+0000 7f7840443700 1 -- 172.21.15.97:0/665427921 shutdown_connections 2023-07-14T11:58:34.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:34.502+0000 7f7840443700 1 --2- 172.21.15.97:0/665427921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7838100610 0x7f7838106ed0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:58:34.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:34.502+0000 7f7840443700 1 --2- 172.21.15.97:0/665427921 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78380fea80 0x7f78380fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:58:34.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:34.502+0000 7f7840443700 1 --2- 172.21.15.97:0/665427921 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78380ffc80 0x7f78381000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:58:34.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:34.502+0000 7f7840443700 1 -- 172.21.15.97:0/665427921 >> 172.21.15.97:0/665427921 conn(0x7f78380fa0b0 msgr2=0x7f78380fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:58:34.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:34.502+0000 7f7840443700 1 -- 172.21.15.97:0/665427921 shutdown_connections 2023-07-14T11:58:34.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:34.503+0000 7f7840443700 1 -- 172.21.15.97:0/665427921 wait complete. 2023-07-14T11:58:34.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:34.503+0000 7f7840443700 1 Processor -- start 2023-07-14T11:58:34.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:34.503+0000 7f7840443700 1 -- start start 2023-07-14T11:58:34.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:34.504+0000 7f7840443700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78380fea80 0x7f7838194880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:58:34.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:34.504+0000 7f7840443700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78380ffc80 0x7f7838194dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:58:34.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:34.504+0000 7f7840443700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7838100610 0x7f783819be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:58:34.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:34.504+0000 7f7840443700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f783819c410 con 0x7f78380ffc80 2023-07-14T11:58:34.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:34.504+0000 7f7840443700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f783819c550 con 0x7f78380fea80 2023-07-14T11:58:34.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:34.504+0000 7f7840443700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f783819c6e0 con 0x7f7838100610 2023-07-14T11:58:34.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:34.504+0000 7f783e9e0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7838100610 0x7f783819be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:58:34.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:34.504+0000 7f783e9e0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7838100610 0x7f783819be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42562/0 (socket says 172.21.15.97:42562) 2023-07-14T11:58:34.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:34.504+0000 7f783e9e0700 1 -- 172.21.15.97:0/220243157 learned_addr learned my addr 172.21.15.97:0/220243157 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-14T11:58:34.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:34.504+0000 7f783e1df700 1 --2- 172.21.15.97:0/220243157 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78380fea80 0x7f7838194880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:58:34.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:34.504+0000 7f783d9de700 1 --2- 172.21.15.97:0/220243157 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78380ffc80 0x7f7838194dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:58:34.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:34.504+0000 7f783e9e0700 1 -- 172.21.15.97:0/220243157 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78380fea80 msgr2=0x7f7838194880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:58:34.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:34.504+0000 7f783e9e0700 1 --2- 172.21.15.97:0/220243157 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78380fea80 0x7f7838194880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:58:34.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:34.504+0000 7f783e9e0700 1 -- 172.21.15.97:0/220243157 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78380ffc80 msgr2=0x7f7838194dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:58:34.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:34.504+0000 7f783e9e0700 1 --2- 172.21.15.97:0/220243157 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78380ffc80 0x7f7838194dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:58:34.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:34.504+0000 7f783e9e0700 1 -- 172.21.15.97:0/220243157 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7830009d00 con 0x7f7838100610 2023-07-14T11:58:34.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:34.504+0000 7f783e9e0700 1 --2- 172.21.15.97:0/220243157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7838100610 0x7f783819be40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7830007a60 tx=0x7f783000bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:58:34.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:34.504+0000 7f782b7fe700 1 -- 172.21.15.97:0/220243157 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f783000e040 con 0x7f7838100610 2023-07-14T11:58:34.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:34.504+0000 7f7840443700 1 -- 172.21.15.97:0/220243157 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7838067fd0 con 0x7f7838100610 2023-07-14T11:58:34.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:34.504+0000 7f782b7fe700 1 -- 172.21.15.97:0/220243157 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f783001e070 con 0x7f7838100610 2023-07-14T11:58:34.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:34.504+0000 7f782b7fe700 1 -- 172.21.15.97:0/220243157 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78300256a0 con 0x7f7838100610 2023-07-14T11:58:34.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:34.504+0000 7f7840443700 1 -- 172.21.15.97:0/220243157 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78380684d0 con 0x7f7838100610 2023-07-14T11:58:34.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:34.505+0000 7f7840443700 1 -- 172.21.15.97:0/220243157 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7838068ec0 con 0x7f7838100610 2023-07-14T11:58:34.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:34.506+0000 7f782b7fe700 1 -- 172.21.15.97:0/220243157 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f783001a4c0 con 0x7f7838100610 2023-07-14T11:58:34.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:34.507+0000 7f782b7fe700 1 --2- 172.21.15.97:0/220243157 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f78240a1150 0x7f78240a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:58:34.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:34.507+0000 7f782b7fe700 1 -- 172.21.15.97:0/220243157 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6723+0+0 (secure 0 0 0) 0x7f7830020070 con 0x7f7838100610 2023-07-14T11:58:34.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:34.507+0000 7f783e1df700 1 --2- 172.21.15.97:0/220243157 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f78240a1150 0x7f78240a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:58:34.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:34.508+0000 7f783e1df700 1 --2- 172.21.15.97:0/220243157 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f78240a1150 0x7f78240a3600 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f78380ffae0 tx=0x7f783400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:58:34.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:34.511+0000 7f782b7fe700 1 -- 172.21.15.97:0/220243157 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7830098ff0 con 0x7f7838100610 2023-07-14T11:58:34.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:34.705+0000 7f7840443700 1 -- 172.21.15.97:0/220243157 --> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f78380686f0 con 0x7f78240a1150 2023-07-14T11:58:34.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:34 smithi097 ceph-mon[129313]: purged_snaps scrub starts 2023-07-14T11:58:34.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:34 smithi097 ceph-mon[129313]: purged_snaps scrub ok 2023-07-14T11:58:34.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:34 smithi097 ceph-mon[129313]: pgmap v226: 1 pgs: 1 remapped+peering; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail 2023-07-14T11:58:34.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:34 smithi097 ceph-mon[129313]: osdmap e73: 11 total, 11 up, 11 in 2023-07-14T11:58:34.894 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:34 smithi017 ceph-mon[126339]: purged_snaps scrub starts 2023-07-14T11:58:34.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:34 smithi017 ceph-mon[126339]: purged_snaps scrub ok 2023-07-14T11:58:34.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:34 smithi017 ceph-mon[126339]: pgmap v226: 1 pgs: 1 remapped+peering; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail 2023-07-14T11:58:34.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:34 smithi017 ceph-mon[126339]: osdmap e73: 11 total, 11 up, 11 in 2023-07-14T11:58:35.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:34 smithi093 ceph-mon[128922]: purged_snaps scrub starts 2023-07-14T11:58:35.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:34 smithi093 ceph-mon[128922]: purged_snaps scrub ok 2023-07-14T11:58:35.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:34 smithi093 ceph-mon[128922]: pgmap v226: 1 pgs: 1 remapped+peering; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail 2023-07-14T11:58:35.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:34 smithi093 ceph-mon[128922]: osdmap e73: 11 total, 11 up, 11 in 2023-07-14T11:58:35.809 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:35 smithi097 ceph-mon[129313]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:58:35.809 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:35 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:58:35.809 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:35 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:58:35.809 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:35 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:58:35.810 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:35 smithi097 ceph-mon[129313]: osdmap e74: 11 total, 11 up, 11 in 2023-07-14T11:58:35.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:35 smithi017 ceph-mon[126339]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:58:35.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:35 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:58:35.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:35 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:58:35.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:35 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:58:35.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:35 smithi017 ceph-mon[126339]: osdmap e74: 11 total, 11 up, 11 in 2023-07-14T11:58:36.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:35 smithi093 ceph-mon[128922]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:58:36.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:35 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T11:58:36.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:35 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T11:58:36.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:35 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:58:36.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:35 smithi093 ceph-mon[128922]: osdmap e74: 11 total, 11 up, 11 in 2023-07-14T11:58:36.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:36 smithi097 ceph-mon[129313]: pgmap v229: 1 pgs: 1 remapped+peering; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail 2023-07-14T11:58:36.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:36 smithi097 ceph-mon[129313]: osdmap e75: 11 total, 11 up, 11 in 2023-07-14T11:58:36.894 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:36 smithi017 ceph-mon[126339]: pgmap v229: 1 pgs: 1 remapped+peering; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail 2023-07-14T11:58:36.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:36 smithi017 ceph-mon[126339]: osdmap e75: 11 total, 11 up, 11 in 2023-07-14T11:58:37.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:36 smithi093 ceph-mon[128922]: pgmap v229: 1 pgs: 1 remapped+peering; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail 2023-07-14T11:58:37.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:36 smithi093 ceph-mon[128922]: osdmap e75: 11 total, 11 up, 11 in 2023-07-14T11:58:37.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:37 smithi097 ceph-mon[129313]: from='client.? 172.21.15.97:0/399080567' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6bbb5b79-8d4d-441f-9583-203df6c78cea"}]: dispatch 2023-07-14T11:58:37.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:37 smithi097 ceph-mon[129313]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6bbb5b79-8d4d-441f-9583-203df6c78cea"}]: dispatch 2023-07-14T11:58:37.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:37 smithi097 ceph-mon[129313]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6bbb5b79-8d4d-441f-9583-203df6c78cea"}]': finished 2023-07-14T11:58:37.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:37 smithi097 ceph-mon[129313]: osdmap e76: 12 total, 11 up, 12 in 2023-07-14T11:58:37.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:37 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T11:58:37.894 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:37 smithi017 ceph-mon[126339]: from='client.? 172.21.15.97:0/399080567' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6bbb5b79-8d4d-441f-9583-203df6c78cea"}]: dispatch 2023-07-14T11:58:37.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:37 smithi017 ceph-mon[126339]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6bbb5b79-8d4d-441f-9583-203df6c78cea"}]: dispatch 2023-07-14T11:58:37.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:37 smithi017 ceph-mon[126339]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6bbb5b79-8d4d-441f-9583-203df6c78cea"}]': finished 2023-07-14T11:58:37.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:37 smithi017 ceph-mon[126339]: osdmap e76: 12 total, 11 up, 12 in 2023-07-14T11:58:37.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:37 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T11:58:38.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:37 smithi093 ceph-mon[128922]: from='client.? 172.21.15.97:0/399080567' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6bbb5b79-8d4d-441f-9583-203df6c78cea"}]: dispatch 2023-07-14T11:58:38.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:37 smithi093 ceph-mon[128922]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6bbb5b79-8d4d-441f-9583-203df6c78cea"}]: dispatch 2023-07-14T11:58:38.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:37 smithi093 ceph-mon[128922]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6bbb5b79-8d4d-441f-9583-203df6c78cea"}]': finished 2023-07-14T11:58:38.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:37 smithi093 ceph-mon[128922]: osdmap e76: 12 total, 11 up, 12 in 2023-07-14T11:58:38.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:37 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T11:58:38.894 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:38 smithi017 ceph-mon[126339]: pgmap v232: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T11:58:38.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:38 smithi017 ceph-mon[126339]: from='client.? 172.21.15.97:0/2304410665' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:58:39.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:38 smithi093 ceph-mon[128922]: pgmap v232: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T11:58:39.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:38 smithi093 ceph-mon[128922]: from='client.? 172.21.15.97:0/2304410665' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:58:39.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:38 smithi097 ceph-mon[129313]: pgmap v232: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T11:58:39.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:38 smithi097 ceph-mon[129313]: from='client.? 172.21.15.97:0/2304410665' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T11:58:41.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:40 smithi093 ceph-mon[128922]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 109 KiB/s, 0 objects/s recovering 2023-07-14T11:58:41.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:40 smithi097 ceph-mon[129313]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 109 KiB/s, 0 objects/s recovering 2023-07-14T11:58:41.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:40 smithi017 ceph-mon[126339]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 109 KiB/s, 0 objects/s recovering 2023-07-14T11:58:43.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:42 smithi093 ceph-mon[128922]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 92 KiB/s, 0 objects/s recovering 2023-07-14T11:58:43.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:42 smithi097 ceph-mon[129313]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 92 KiB/s, 0 objects/s recovering 2023-07-14T11:58:43.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:42 smithi017 ceph-mon[126339]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 92 KiB/s, 0 objects/s recovering 2023-07-14T11:58:44.607 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:44 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-07-14T11:58:44.608 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:44 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:58:44.608 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:44 smithi097 ceph-mon[129313]: Deploying daemon osd.11 on smithi097 2023-07-14T11:58:44.608 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:44 smithi097 ceph-mon[129313]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-14T11:58:44.861 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:44 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-07-14T11:58:44.861 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:44 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:58:44.861 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:44 smithi093 ceph-mon[128922]: Deploying daemon osd.11 on smithi097 2023-07-14T11:58:44.862 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:44 smithi093 ceph-mon[128922]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-14T11:58:44.894 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:44 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-07-14T11:58:44.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:44 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:58:44.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:44 smithi017 ceph-mon[126339]: Deploying daemon osd.11 on smithi097 2023-07-14T11:58:44.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:44 smithi017 ceph-mon[126339]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-14T11:58:46.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:46 smithi097 ceph-mon[129313]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2023-07-14T11:58:47.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:46 smithi093 ceph-mon[128922]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2023-07-14T11:58:47.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:46 smithi017 ceph-mon[126339]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2023-07-14T11:58:48.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:48 smithi097 ceph-mon[129313]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2023-07-14T11:58:48.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:48 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:48.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:48 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:48.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:48 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:58:49.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:48 smithi093 ceph-mon[128922]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2023-07-14T11:58:49.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:48 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:49.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:48 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:49.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:48 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:58:49.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:48 smithi017 ceph-mon[126339]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2023-07-14T11:58:49.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:48 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:49.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:48 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:49.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:48 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:58:51.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:51.025+0000 7f782b7fe700 1 -- 172.21.15.97:0/220243157 <== mgr.14152 v2:172.21.15.17:6800/4237449858 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f78380686f0 con 0x7f78240a1150 2023-07-14T11:58:51.007 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 11 on host 'smithi097' 2023-07-14T11:58:51.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:51.027+0000 7f7840443700 1 -- 172.21.15.97:0/220243157 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f78240a1150 msgr2=0x7f78240a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:58:51.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:51.027+0000 7f7840443700 1 --2- 172.21.15.97:0/220243157 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f78240a1150 0x7f78240a3600 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f78380ffae0 tx=0x7f783400b410 comp rx=0 tx=0).stop 2023-07-14T11:58:51.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:51.027+0000 7f7840443700 1 -- 172.21.15.97:0/220243157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7838100610 msgr2=0x7f783819be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:58:51.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:51.027+0000 7f7840443700 1 --2- 172.21.15.97:0/220243157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7838100610 0x7f783819be40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7830007a60 tx=0x7f783000bd60 comp rx=0 tx=0).stop 2023-07-14T11:58:51.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:51.028+0000 7f7840443700 1 -- 172.21.15.97:0/220243157 shutdown_connections 2023-07-14T11:58:51.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:51.028+0000 7f7840443700 1 --2- 172.21.15.97:0/220243157 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f78240a1150 0x7f78240a3600 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:58:51.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:51.028+0000 7f7840443700 1 --2- 172.21.15.97:0/220243157 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78380fea80 0x7f7838194880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:58:51.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:51.028+0000 7f7840443700 1 --2- 172.21.15.97:0/220243157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7838100610 0x7f783819be40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:58:51.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:51.028+0000 7f7840443700 1 --2- 172.21.15.97:0/220243157 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78380ffc80 0x7f7838194dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:58:51.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:51.028+0000 7f7840443700 1 -- 172.21.15.97:0/220243157 >> 172.21.15.97:0/220243157 conn(0x7f78380fa0b0 msgr2=0x7f7838103100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:58:51.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:51.028+0000 7f7840443700 1 -- 172.21.15.97:0/220243157 shutdown_connections 2023-07-14T11:58:51.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:58:51.028+0000 7f7840443700 1 -- 172.21.15.97:0/220243157 wait complete. 2023-07-14T11:58:51.032 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:50 smithi097 ceph-mon[129313]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-14T11:58:51.033 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:50 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:51.033 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:50 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:51.033 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:50 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:58:51.033 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:50 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:58:51.033 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:50 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:51.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:50 smithi093 ceph-mon[128922]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-14T11:58:51.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:50 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:51.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:50 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:51.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:50 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:58:51.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:50 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:58:51.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:50 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:51.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:50 smithi017 ceph-mon[126339]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-14T11:58:51.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:50 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:51.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:50 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:51.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:50 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:58:51.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:50 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:58:51.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:50 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:51.334 DEBUG:teuthology.orchestra.run.smithi097:osd.11> sudo journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@osd.11.service 2023-07-14T11:58:51.337 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-07-14T11:58:51.337 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph osd stat -f json 2023-07-14T11:58:51.372 INFO:journalctl@ceph.osd.11.smithi097.stdout:-- Logs begin at Fri 2023-07-14 11:24:16 UTC. -- 2023-07-14T11:58:52.083 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T11:58:52.273 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:52 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:52.273 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:52 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:52.273 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:52 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:58:52.274 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 11:58:52 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T11:58:52.108+0000 7f8e1ae0c640 -1 osd.11 0 log_to_monitors true 2023-07-14T11:58:52.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:52 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:52.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:52 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:52.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:52 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:58:52.394 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:52 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:52.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:52 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:52.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:52 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:58:53.328 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:53 smithi097 ceph-mon[129313]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T11:58:53.328 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:53 smithi097 ceph-mon[129313]: from='osd.11 [v2:172.21.15.97:6824/3389913754,v1:172.21.15.97:6825/3389913754]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-14T11:58:53.328 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:53 smithi097 ceph-mon[129313]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-14T11:58:53.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:53 smithi093 ceph-mon[128922]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T11:58:53.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:53 smithi093 ceph-mon[128922]: from='osd.11 [v2:172.21.15.97:6824/3389913754,v1:172.21.15.97:6825/3389913754]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-14T11:58:53.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:53 smithi093 ceph-mon[128922]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-14T11:58:53.394 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:53 smithi017 ceph-mon[126339]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T11:58:53.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:53 smithi017 ceph-mon[126339]: from='osd.11 [v2:172.21.15.97:6824/3389913754,v1:172.21.15.97:6825/3389913754]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-14T11:58:53.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:53 smithi017 ceph-mon[126339]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-14T11:58:53.747 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.766+0000 7ff0e0a0a700 1 -- 172.21.15.17:0/2121726725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0dc074540 msgr2=0x7ff0dc0729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:58:53.748 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.766+0000 7ff0e0a0a700 1 --2- 172.21.15.17:0/2121726725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0dc074540 0x7ff0dc0729a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc0098f0 tx=0x7ff0cc009c00 comp rx=0 tx=0).stop 2023-07-14T11:58:53.748 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.766+0000 7ff0e0a0a700 1 -- 172.21.15.17:0/2121726725 shutdown_connections 2023-07-14T11:58:53.748 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.766+0000 7ff0e0a0a700 1 --2- 172.21.15.17:0/2121726725 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0dc107a60 0x7ff0dc109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:58:53.748 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.766+0000 7ff0e0a0a700 1 --2- 172.21.15.17:0/2121726725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0dc074540 0x7ff0dc0729a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:58:53.748 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.766+0000 7ff0e0a0a700 1 --2- 172.21.15.17:0/2121726725 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0dc072f70 0x7ff0dc0733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:58:53.748 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.766+0000 7ff0e0a0a700 1 -- 172.21.15.17:0/2121726725 >> 172.21.15.17:0/2121726725 conn(0x7ff0dc0fb1a0 msgr2=0x7ff0dc0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:58:53.749 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.766+0000 7ff0e0a0a700 1 -- 172.21.15.17:0/2121726725 shutdown_connections 2023-07-14T11:58:53.749 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.766+0000 7ff0e0a0a700 1 -- 172.21.15.17:0/2121726725 wait complete. 2023-07-14T11:58:53.750 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.767+0000 7ff0e0a0a700 1 Processor -- start 2023-07-14T11:58:53.750 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.768+0000 7ff0e0a0a700 1 -- start start 2023-07-14T11:58:53.750 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.768+0000 7ff0e0a0a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0dc072f70 0x7ff0dc197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:58:53.750 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.768+0000 7ff0e0a0a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0dc074540 0x7ff0dc197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:58:53.750 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.768+0000 7ff0e0a0a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0dc107a60 0x7ff0dc19f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:58:53.750 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.768+0000 7ff0e0a0a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0dc19f630 con 0x7ff0dc074540 2023-07-14T11:58:53.751 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.768+0000 7ff0e0a0a700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0dc19f770 con 0x7ff0dc072f70 2023-07-14T11:58:53.751 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.768+0000 7ff0e0a0a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0dc19f900 con 0x7ff0dc107a60 2023-07-14T11:58:53.751 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.768+0000 7ff0d9d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0dc074540 0x7ff0dc197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:58:53.751 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.768+0000 7ff0d9d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0dc074540 0x7ff0dc197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45358/0 (socket says 172.21.15.17:45358) 2023-07-14T11:58:53.751 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.768+0000 7ff0d9d9b700 1 -- 172.21.15.17:0/4212030770 learned_addr learned my addr 172.21.15.17:0/4212030770 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:58:53.751 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.768+0000 7ff0dad9d700 1 --2- 172.21.15.17:0/4212030770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0dc107a60 0x7ff0dc19f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:58:53.751 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.769+0000 7ff0da59c700 1 --2- 172.21.15.17:0/4212030770 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0dc072f70 0x7ff0dc197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:58:53.752 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.769+0000 7ff0dad9d700 1 -- 172.21.15.17:0/4212030770 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0dc072f70 msgr2=0x7ff0dc197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:58:53.752 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.769+0000 7ff0dad9d700 1 --2- 172.21.15.17:0/4212030770 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0dc072f70 0x7ff0dc197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:58:53.752 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.769+0000 7ff0dad9d700 1 -- 172.21.15.17:0/4212030770 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0dc074540 msgr2=0x7ff0dc197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:58:53.752 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.769+0000 7ff0dad9d700 1 --2- 172.21.15.17:0/4212030770 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0dc074540 0x7ff0dc197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:58:53.752 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.769+0000 7ff0dad9d700 1 -- 172.21.15.17:0/4212030770 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0cc009520 con 0x7ff0dc107a60 2023-07-14T11:58:53.752 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.769+0000 7ff0d9d9b700 1 --2- 172.21.15.17:0/4212030770 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0dc074540 0x7ff0dc197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:58:53.753 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.769+0000 7ff0dad9d700 1 --2- 172.21.15.17:0/4212030770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0dc107a60 0x7ff0dc19f060 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4009f60 tx=0x7ff0d4009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:58:53.753 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.769+0000 7ff0cb7fe700 1 -- 172.21.15.17:0/4212030770 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d4018070 con 0x7ff0dc107a60 2023-07-14T11:58:53.753 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.769+0000 7ff0cb7fe700 1 -- 172.21.15.17:0/4212030770 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0d400f040 con 0x7ff0dc107a60 2023-07-14T11:58:53.753 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.769+0000 7ff0e0a0a700 1 -- 172.21.15.17:0/4212030770 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0dc19fb80 con 0x7ff0dc107a60 2023-07-14T11:58:53.754 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.769+0000 7ff0cb7fe700 1 -- 172.21.15.17:0/4212030770 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d400ccc0 con 0x7ff0dc107a60 2023-07-14T11:58:53.754 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.770+0000 7ff0e0a0a700 1 -- 172.21.15.17:0/4212030770 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0dc1a0080 con 0x7ff0dc107a60 2023-07-14T11:58:53.754 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.771+0000 7ff0e0a0a700 1 -- 172.21.15.17:0/4212030770 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0dc106110 con 0x7ff0dc107a60 2023-07-14T11:58:53.754 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.773+0000 7ff0cb7fe700 1 -- 172.21.15.17:0/4212030770 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff0d4003a00 con 0x7ff0dc107a60 2023-07-14T11:58:53.755 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.774+0000 7ff0cb7fe700 1 --2- 172.21.15.17:0/4212030770 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7ff0c40a11d0 0x7ff0c40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:58:53.755 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.774+0000 7ff0cb7fe700 1 -- 172.21.15.17:0/4212030770 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6831+0+0 (secure 0 0 0) 0x7ff0d40cbe30 con 0x7ff0dc107a60 2023-07-14T11:58:53.755 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.774+0000 7ff0da59c700 1 --2- 172.21.15.17:0/4212030770 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7ff0c40a11d0 0x7ff0c40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:58:53.756 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.775+0000 7ff0da59c700 1 --2- 172.21.15.17:0/4212030770 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7ff0c40a11d0 0x7ff0c40a3680 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc00fa40 tx=0x7ff0cc019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:58:53.759 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.777+0000 7ff0cb7fe700 1 -- 172.21.15.17:0/4212030770 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff0d4095b40 con 0x7ff0dc107a60 2023-07-14T11:58:53.974 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.993+0000 7ff0e0a0a700 1 -- 172.21.15.17:0/4212030770 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7ff0dc0611c0 con 0x7ff0dc107a60 2023-07-14T11:58:53.974 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.993+0000 7ff0cb7fe700 1 -- 172.21.15.17:0/4212030770 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v77) v1 ==== 74+0+133 (secure 0 0 0) 0x7ff0d4099160 con 0x7ff0dc107a60 2023-07-14T11:58:53.975 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:58:53.977 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.996+0000 7ff0e0a0a700 1 -- 172.21.15.17:0/4212030770 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7ff0c40a11d0 msgr2=0x7ff0c40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:58:53.977 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.996+0000 7ff0e0a0a700 1 --2- 172.21.15.17:0/4212030770 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7ff0c40a11d0 0x7ff0c40a3680 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc00fa40 tx=0x7ff0cc019040 comp rx=0 tx=0).stop 2023-07-14T11:58:53.977 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.996+0000 7ff0e0a0a700 1 -- 172.21.15.17:0/4212030770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0dc107a60 msgr2=0x7ff0dc19f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:58:53.977 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.996+0000 7ff0e0a0a700 1 --2- 172.21.15.17:0/4212030770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0dc107a60 0x7ff0dc19f060 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4009f60 tx=0x7ff0d4009f90 comp rx=0 tx=0).stop 2023-07-14T11:58:53.978 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.996+0000 7ff0e0a0a700 1 -- 172.21.15.17:0/4212030770 shutdown_connections 2023-07-14T11:58:53.978 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.996+0000 7ff0e0a0a700 1 --2- 172.21.15.17:0/4212030770 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7ff0c40a11d0 0x7ff0c40a3680 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:58:53.978 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.996+0000 7ff0e0a0a700 1 --2- 172.21.15.17:0/4212030770 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0dc072f70 0x7ff0dc197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:58:53.978 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.996+0000 7ff0e0a0a700 1 --2- 172.21.15.17:0/4212030770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0dc107a60 0x7ff0dc19f060 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:58:53.978 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.996+0000 7ff0e0a0a700 1 --2- 172.21.15.17:0/4212030770 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0dc074540 0x7ff0dc197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:58:53.978 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.996+0000 7ff0e0a0a700 1 -- 172.21.15.17:0/4212030770 >> 172.21.15.17:0/4212030770 conn(0x7ff0dc0fb1a0 msgr2=0x7ff0dc0fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:58:53.978 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.996+0000 7ff0e0a0a700 1 -- 172.21.15.17:0/4212030770 shutdown_connections 2023-07-14T11:58:53.978 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:53.996+0000 7ff0e0a0a700 1 -- 172.21.15.17:0/4212030770 wait complete. 2023-07-14T11:58:54.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:54 smithi093 ceph-mon[128922]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-07-14T11:58:54.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:54 smithi093 ceph-mon[128922]: osdmap e77: 12 total, 11 up, 12 in 2023-07-14T11:58:54.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:54 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T11:58:54.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:54 smithi093 ceph-mon[128922]: from='osd.11 [v2:172.21.15.97:6824/3389913754,v1:172.21.15.97:6825/3389913754]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-07-14T11:58:54.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:54 smithi093 ceph-mon[128922]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-07-14T11:58:54.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:54 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:54.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:54 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:54.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:54 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:58:54.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:54 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:58:54.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:54 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:58:54.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:54 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:58:54.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:54 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:54.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:54 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:58:54.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:54 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:58:54.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:54 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:54.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:54 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/4212030770' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-14T11:58:54.384 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 11:58:54 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T11:58:54.040+0000 7f8e10484700 -1 osd.11 0 waiting for initial osdmap 2023-07-14T11:58:54.385 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 11:58:54 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T11:58:54.051+0000 7f8e09a74700 -1 osd.11 78 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T11:58:54.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:54 smithi097 ceph-mon[129313]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-07-14T11:58:54.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:54 smithi097 ceph-mon[129313]: osdmap e77: 12 total, 11 up, 12 in 2023-07-14T11:58:54.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:54 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T11:58:54.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:54 smithi097 ceph-mon[129313]: from='osd.11 [v2:172.21.15.97:6824/3389913754,v1:172.21.15.97:6825/3389913754]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-07-14T11:58:54.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:54 smithi097 ceph-mon[129313]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-07-14T11:58:54.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:54 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:54.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:54 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:54.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:54 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:58:54.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:54 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:58:54.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:54 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:58:54.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:54 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:58:54.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:54 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:54.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:54 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:58:54.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:54 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:58:54.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:54 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:54.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:54 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/4212030770' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-14T11:58:54.394 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:54 smithi017 ceph-mon[126339]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-07-14T11:58:54.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:54 smithi017 ceph-mon[126339]: osdmap e77: 12 total, 11 up, 12 in 2023-07-14T11:58:54.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:54 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T11:58:54.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:54 smithi017 ceph-mon[126339]: from='osd.11 [v2:172.21.15.97:6824/3389913754,v1:172.21.15.97:6825/3389913754]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-07-14T11:58:54.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:54 smithi017 ceph-mon[126339]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-07-14T11:58:54.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:54 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:54.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:54 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:54.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:54 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:58:54.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:54 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:58:54.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:54 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:58:54.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:54 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-14T11:58:54.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:54 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:54.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:54 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:58:54.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:54 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:58:54.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:54 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:58:54.397 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:54 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/4212030770' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-14T11:58:54.563 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":77,"num_osds":12,"num_up_osds":11,"osd_up_since":1689335913,"num_in_osds":12,"osd_in_since":1689335917,"num_remapped_pgs":0} 2023-07-14T11:58:55.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:55 smithi093 ceph-mon[128922]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T11:58:55.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:55 smithi093 ceph-mon[128922]: Detected new or changed devices on smithi097 2023-07-14T11:58:55.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:55 smithi093 ceph-mon[128922]: Adjusting osd_memory_target on smithi097 to 4292M 2023-07-14T11:58:55.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:55 smithi093 ceph-mon[128922]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2023-07-14T11:58:55.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:55 smithi093 ceph-mon[128922]: osdmap e78: 12 total, 11 up, 12 in 2023-07-14T11:58:55.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:55 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T11:58:55.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:55 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T11:58:55.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:55 smithi097 ceph-mon[129313]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T11:58:55.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:55 smithi097 ceph-mon[129313]: Detected new or changed devices on smithi097 2023-07-14T11:58:55.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:55 smithi097 ceph-mon[129313]: Adjusting osd_memory_target on smithi097 to 4292M 2023-07-14T11:58:55.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:55 smithi097 ceph-mon[129313]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2023-07-14T11:58:55.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:55 smithi097 ceph-mon[129313]: osdmap e78: 12 total, 11 up, 12 in 2023-07-14T11:58:55.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:55 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T11:58:55.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:55 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T11:58:55.394 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:55 smithi017 ceph-mon[126339]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T11:58:55.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:55 smithi017 ceph-mon[126339]: Detected new or changed devices on smithi097 2023-07-14T11:58:55.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:55 smithi017 ceph-mon[126339]: Adjusting osd_memory_target on smithi097 to 4292M 2023-07-14T11:58:55.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:55 smithi017 ceph-mon[126339]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2023-07-14T11:58:55.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:55 smithi017 ceph-mon[126339]: osdmap e78: 12 total, 11 up, 12 in 2023-07-14T11:58:55.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:55 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T11:58:55.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:55 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T11:58:55.564 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph osd stat -f json 2023-07-14T11:58:56.307 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T11:58:56.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:56 smithi093 ceph-mon[128922]: purged_snaps scrub starts 2023-07-14T11:58:56.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:56 smithi093 ceph-mon[128922]: purged_snaps scrub ok 2023-07-14T11:58:56.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:56 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T11:58:56.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:56 smithi093 ceph-mon[128922]: osd.11 [v2:172.21.15.97:6824/3389913754,v1:172.21.15.97:6825/3389913754] boot 2023-07-14T11:58:56.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:56 smithi093 ceph-mon[128922]: osdmap e79: 12 total, 12 up, 12 in 2023-07-14T11:58:56.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:56 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T11:58:56.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:56 smithi097 ceph-mon[129313]: purged_snaps scrub starts 2023-07-14T11:58:56.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:56 smithi097 ceph-mon[129313]: purged_snaps scrub ok 2023-07-14T11:58:56.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:56 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T11:58:56.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:56 smithi097 ceph-mon[129313]: osd.11 [v2:172.21.15.97:6824/3389913754,v1:172.21.15.97:6825/3389913754] boot 2023-07-14T11:58:56.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:56 smithi097 ceph-mon[129313]: osdmap e79: 12 total, 12 up, 12 in 2023-07-14T11:58:56.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:56 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T11:58:56.394 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:56 smithi017 ceph-mon[126339]: purged_snaps scrub starts 2023-07-14T11:58:56.394 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:56 smithi017 ceph-mon[126339]: purged_snaps scrub ok 2023-07-14T11:58:56.394 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:56 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T11:58:56.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:56 smithi017 ceph-mon[126339]: osd.11 [v2:172.21.15.97:6824/3389913754,v1:172.21.15.97:6825/3389913754] boot 2023-07-14T11:58:56.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:56 smithi017 ceph-mon[126339]: osdmap e79: 12 total, 12 up, 12 in 2023-07-14T11:58:56.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:56 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T11:58:57.205 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:57 smithi017 ceph-mon[126339]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T11:58:57.206 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:57 smithi017 ceph-mon[126339]: osdmap e80: 12 total, 12 up, 12 in 2023-07-14T11:58:57.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:57 smithi093 ceph-mon[128922]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T11:58:57.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:57 smithi093 ceph-mon[128922]: osdmap e80: 12 total, 12 up, 12 in 2023-07-14T11:58:57.384 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:57 smithi097 ceph-mon[129313]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T11:58:57.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:57 smithi097 ceph-mon[129313]: osdmap e80: 12 total, 12 up, 12 in 2023-07-14T11:58:57.841 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:57.859+0000 7f35647f0700 1 -- 172.21.15.17:0/2399486664 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f355c100d90 msgr2=0x7f355c1011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:58:57.842 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:57.859+0000 7f35647f0700 1 --2- 172.21.15.17:0/2399486664 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f355c100d90 0x7f355c1011e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f354c009990 tx=0x7f354c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:58:57.842 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:57.860+0000 7f35647f0700 1 -- 172.21.15.17:0/2399486664 shutdown_connections 2023-07-14T11:58:57.842 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:57.860+0000 7f35647f0700 1 --2- 172.21.15.17:0/2399486664 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f355c0ffb90 0x7f355c0fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:58:57.842 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:57.860+0000 7f35647f0700 1 --2- 172.21.15.17:0/2399486664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f355c101720 0x7f355c107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:58:57.842 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:57.860+0000 7f35647f0700 1 --2- 172.21.15.17:0/2399486664 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f355c100d90 0x7f355c1011e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:58:57.842 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:57.860+0000 7f35647f0700 1 -- 172.21.15.17:0/2399486664 >> 172.21.15.17:0/2399486664 conn(0x7f355c0fb1a0 msgr2=0x7f355c0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:58:57.843 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:57.860+0000 7f35647f0700 1 -- 172.21.15.17:0/2399486664 shutdown_connections 2023-07-14T11:58:57.843 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:57.860+0000 7f35647f0700 1 -- 172.21.15.17:0/2399486664 wait complete. 2023-07-14T11:58:57.844 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:57.861+0000 7f35647f0700 1 Processor -- start 2023-07-14T11:58:57.844 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:57.861+0000 7f35647f0700 1 -- start start 2023-07-14T11:58:57.845 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:57.861+0000 7f35647f0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f355c0ffb90 0x7f355c197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:58:57.845 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:57.861+0000 7f35647f0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f355c100d90 0x7f355c197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:58:57.845 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:57.861+0000 7f35647f0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f355c101720 0x7f355c19f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:58:57.845 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:57.861+0000 7f35647f0700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355c19f630 con 0x7f355c100d90 2023-07-14T11:58:57.845 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:57.861+0000 7f35647f0700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355c19f770 con 0x7f355c0ffb90 2023-07-14T11:58:57.845 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:57.861+0000 7f35647f0700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355c19f900 con 0x7f355c101720 2023-07-14T11:58:57.846 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:57.862+0000 7f356258c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f355c0ffb90 0x7f355c197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:58:57.846 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:57.862+0000 7f3561d8b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f355c100d90 0x7f355c197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:58:57.846 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:57.862+0000 7f356258c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f355c0ffb90 0x7f355c197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.17:49702/0 (socket says 172.21.15.17:49702) 2023-07-14T11:58:57.846 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:57.862+0000 7f356258c700 1 -- 172.21.15.17:0/2740040811 learned_addr learned my addr 172.21.15.17:0/2740040811 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:58:57.846 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:57.862+0000 7f3562d8d700 1 --2- 172.21.15.17:0/2740040811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f355c101720 0x7f355c19f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:58:57.846 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:57.862+0000 7f356258c700 1 -- 172.21.15.17:0/2740040811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f355c101720 msgr2=0x7f355c19f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:58:57.846 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:57.862+0000 7f356258c700 1 --2- 172.21.15.17:0/2740040811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f355c101720 0x7f355c19f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:58:57.847 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:57.862+0000 7f356258c700 1 -- 172.21.15.17:0/2740040811 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f355c100d90 msgr2=0x7f355c197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:58:57.847 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:57.862+0000 7f356258c700 1 --2- 172.21.15.17:0/2740040811 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f355c100d90 0x7f355c197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:58:57.847 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:57.862+0000 7f356258c700 1 -- 172.21.15.17:0/2740040811 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f354c0095f0 con 0x7f355c0ffb90 2023-07-14T11:58:57.847 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:57.862+0000 7f356258c700 1 --2- 172.21.15.17:0/2740040811 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f355c0ffb90 0x7f355c197aa0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f355800e970 tx=0x7f355800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:58:57.847 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:57.862+0000 7f35537fe700 1 -- 172.21.15.17:0/2740040811 <== mon.2 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3558009790 con 0x7f355c0ffb90 2023-07-14T11:58:57.847 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:57.863+0000 7f35537fe700 1 -- 172.21.15.17:0/2740040811 <== mon.2 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f355800c7e0 con 0x7f355c0ffb90 2023-07-14T11:58:57.847 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:57.863+0000 7f35537fe700 1 -- 172.21.15.17:0/2740040811 <== mon.2 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3558003c70 con 0x7f355c0ffb90 2023-07-14T11:58:57.848 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:57.863+0000 7f35647f0700 1 -- 172.21.15.17:0/2740040811 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f355c19fbe0 con 0x7f355c0ffb90 2023-07-14T11:58:57.848 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:57.863+0000 7f35647f0700 1 -- 172.21.15.17:0/2740040811 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f355c1a0090 con 0x7f355c0ffb90 2023-07-14T11:58:57.849 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:57.864+0000 7f35647f0700 1 -- 172.21.15.17:0/2740040811 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f355c192210 con 0x7f355c0ffb90 2023-07-14T11:58:57.849 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:57.866+0000 7f35537fe700 1 -- 172.21.15.17:0/2740040811 <== mon.2 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f355800c950 con 0x7f355c0ffb90 2023-07-14T11:58:57.850 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:57.867+0000 7f35537fe700 1 --2- 172.21.15.17:0/2740040811 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f35480a11d0 0x7f35480a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:58:57.850 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:57.867+0000 7f35537fe700 1 -- 172.21.15.17:0/2740040811 <== mon.2 v2:172.21.15.93:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3558014070 con 0x7f355c0ffb90 2023-07-14T11:58:57.850 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:57.868+0000 7f3561d8b700 1 --2- 172.21.15.17:0/2740040811 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f35480a11d0 0x7f35480a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:58:57.850 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:57.868+0000 7f3561d8b700 1 --2- 172.21.15.17:0/2740040811 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f35480a11d0 0x7f35480a3680 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f354c00b680 tx=0x7f354c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:58:57.851 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:57.869+0000 7f35537fe700 1 -- 172.21.15.17:0/2740040811 <== mon.2 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3558096320 con 0x7f355c0ffb90 2023-07-14T11:58:58.061 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:58.080+0000 7f35647f0700 1 -- 172.21.15.17:0/2740040811 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f355c02c8a0 con 0x7f355c0ffb90 2023-07-14T11:58:58.061 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:58.080+0000 7f35537fe700 1 -- 172.21.15.17:0/2740040811 <== mon.2 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) v1 ==== 74+0+133 (secure 0 0 0) 0x7f355801f340 con 0x7f355c0ffb90 2023-07-14T11:58:58.062 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:58:58.064 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:58.083+0000 7f35647f0700 1 -- 172.21.15.17:0/2740040811 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f35480a11d0 msgr2=0x7f35480a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:58:58.064 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:58.083+0000 7f35647f0700 1 --2- 172.21.15.17:0/2740040811 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f35480a11d0 0x7f35480a3680 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f354c00b680 tx=0x7f354c005fd0 comp rx=0 tx=0).stop 2023-07-14T11:58:58.065 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:58.083+0000 7f35647f0700 1 -- 172.21.15.17:0/2740040811 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f355c0ffb90 msgr2=0x7f355c197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:58:58.065 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:58.083+0000 7f35647f0700 1 --2- 172.21.15.17:0/2740040811 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f355c0ffb90 0x7f355c197aa0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f355800e970 tx=0x7f355800ec80 comp rx=0 tx=0).stop 2023-07-14T11:58:58.065 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:58.083+0000 7f35647f0700 1 -- 172.21.15.17:0/2740040811 shutdown_connections 2023-07-14T11:58:58.065 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:58.083+0000 7f35647f0700 1 --2- 172.21.15.17:0/2740040811 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f35480a11d0 0x7f35480a3680 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:58:58.065 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:58.083+0000 7f35647f0700 1 --2- 172.21.15.17:0/2740040811 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f355c0ffb90 0x7f355c197aa0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:58:58.065 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:58.083+0000 7f35647f0700 1 --2- 172.21.15.17:0/2740040811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f355c101720 0x7f355c19f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:58:58.065 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:58.083+0000 7f35647f0700 1 --2- 172.21.15.17:0/2740040811 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f355c100d90 0x7f355c197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:58:58.066 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:58.083+0000 7f35647f0700 1 -- 172.21.15.17:0/2740040811 >> 172.21.15.17:0/2740040811 conn(0x7f355c0fb1a0 msgr2=0x7f355c1066d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:58:58.066 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:58.083+0000 7f35647f0700 1 -- 172.21.15.17:0/2740040811 shutdown_connections 2023-07-14T11:58:58.066 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:58:58.083+0000 7f35647f0700 1 -- 172.21.15.17:0/2740040811 wait complete. 2023-07-14T11:58:58.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:58 smithi093 ceph-mon[128922]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:58:58.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:58 smithi097 ceph-mon[129313]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:58:58.394 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:58 smithi017 ceph-mon[126339]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:58:58.587 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1689335935,"num_in_osds":12,"osd_in_since":1689335917,"num_remapped_pgs":0} 2023-07-14T11:58:58.588 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph osd dump --format=json 2023-07-14T11:58:59.327 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T11:58:59.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:58:59 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/2740040811' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-14T11:58:59.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:58:59 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/2740040811' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-14T11:58:59.394 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:58:59 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/2740040811' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-14T11:59:00.242 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:00 smithi017 ceph-mon[126339]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:00.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:00 smithi093 ceph-mon[128922]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:00.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:00 smithi097 ceph-mon[129313]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:00.990 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.008+0000 7fa8f1882700 1 -- 172.21.15.17:0/3071418243 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8ec104ea0 msgr2=0x7fa8ec10ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:00.990 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.008+0000 7fa8f1882700 1 --2- 172.21.15.17:0/3071418243 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8ec104ea0 0x7fa8ec10ca20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa8e400b0d0 tx=0x7fa8e400b3e0 comp rx=0 tx=0).stop 2023-07-14T11:59:00.990 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.008+0000 7fa8f1882700 1 -- 172.21.15.17:0/3071418243 shutdown_connections 2023-07-14T11:59:00.990 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.008+0000 7fa8f1882700 1 --2- 172.21.15.17:0/3071418243 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8ec104ea0 0x7fa8ec10ca20 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:00.990 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.008+0000 7fa8f1882700 1 --2- 172.21.15.17:0/3071418243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8ec0ffe00 0x7fa8ec1001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:00.991 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.008+0000 7fa8f1882700 1 --2- 172.21.15.17:0/3071418243 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa8ec1007a0 0x7fa8ec104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:00.991 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.008+0000 7fa8f1882700 1 -- 172.21.15.17:0/3071418243 >> 172.21.15.17:0/3071418243 conn(0x7fa8ec0fb660 msgr2=0x7fa8ec0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:00.991 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.008+0000 7fa8f1882700 1 -- 172.21.15.17:0/3071418243 shutdown_connections 2023-07-14T11:59:00.991 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.008+0000 7fa8f1882700 1 -- 172.21.15.17:0/3071418243 wait complete. 2023-07-14T11:59:00.992 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.009+0000 7fa8f1882700 1 Processor -- start 2023-07-14T11:59:00.993 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.010+0000 7fa8f1882700 1 -- start start 2023-07-14T11:59:00.993 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.010+0000 7fa8f1882700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8ec0ffe00 0x7fa8ec072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:00.993 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.010+0000 7fa8f1882700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa8ec1007a0 0x7fa8ec072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:00.993 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.010+0000 7fa8f1882700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8ec104ea0 0x7fa8ec0732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:00.993 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.010+0000 7fa8f1882700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ec1a4220 con 0x7fa8ec104ea0 2023-07-14T11:59:00.993 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.010+0000 7fa8f1882700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ec1a4390 con 0x7fa8ec1007a0 2023-07-14T11:59:00.993 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.010+0000 7fa8f1882700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ec1a4550 con 0x7fa8ec0ffe00 2023-07-14T11:59:00.994 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.010+0000 7fa8eb7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8ec104ea0 0x7fa8ec0732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:00.994 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.010+0000 7fa8eb7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8ec104ea0 0x7fa8ec0732d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45402/0 (socket says 172.21.15.17:45402) 2023-07-14T11:59:00.994 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.010+0000 7fa8eb7fe700 1 -- 172.21.15.17:0/1736198887 learned_addr learned my addr 172.21.15.17:0/1736198887 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:59:00.994 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.010+0000 7fa8eaffd700 1 --2- 172.21.15.17:0/1736198887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8ec0ffe00 0x7fa8ec072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:00.994 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.010+0000 7fa8ea7fc700 1 --2- 172.21.15.17:0/1736198887 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa8ec1007a0 0x7fa8ec072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:00.994 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.011+0000 7fa8eb7fe700 1 -- 172.21.15.17:0/1736198887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8ec0ffe00 msgr2=0x7fa8ec072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:00.994 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.011+0000 7fa8eb7fe700 1 --2- 172.21.15.17:0/1736198887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8ec0ffe00 0x7fa8ec072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:00.994 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.011+0000 7fa8eb7fe700 1 -- 172.21.15.17:0/1736198887 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa8ec1007a0 msgr2=0x7fa8ec072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:00.995 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.011+0000 7fa8eb7fe700 1 --2- 172.21.15.17:0/1736198887 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa8ec1007a0 0x7fa8ec072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:00.995 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.011+0000 7fa8eb7fe700 1 -- 172.21.15.17:0/1736198887 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e4009d00 con 0x7fa8ec104ea0 2023-07-14T11:59:00.995 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.011+0000 7fa8eb7fe700 1 --2- 172.21.15.17:0/1736198887 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8ec104ea0 0x7fa8ec0732d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4007890 tx=0x7fa8e40078c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:00.995 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.011+0000 7fa8f0880700 1 -- 172.21.15.17:0/1736198887 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e400e040 con 0x7fa8ec104ea0 2023-07-14T11:59:00.995 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.011+0000 7fa8f0880700 1 -- 172.21.15.17:0/1736198887 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8e400bd30 con 0x7fa8ec104ea0 2023-07-14T11:59:00.995 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.011+0000 7fa8f0880700 1 -- 172.21.15.17:0/1736198887 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e401d470 con 0x7fa8ec104ea0 2023-07-14T11:59:00.995 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.011+0000 7fa8f1882700 1 -- 172.21.15.17:0/1736198887 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8ec1a47d0 con 0x7fa8ec104ea0 2023-07-14T11:59:00.996 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.012+0000 7fa8f1882700 1 -- 172.21.15.17:0/1736198887 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8ec1a4bf0 con 0x7fa8ec104ea0 2023-07-14T11:59:00.996 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.013+0000 7fa8f1882700 1 -- 172.21.15.17:0/1736198887 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8ec04e210 con 0x7fa8ec104ea0 2023-07-14T11:59:00.996 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.014+0000 7fa8f0880700 1 -- 172.21.15.17:0/1736198887 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa8e4004a50 con 0x7fa8ec104ea0 2023-07-14T11:59:00.997 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.016+0000 7fa8f0880700 1 --2- 172.21.15.17:0/1736198887 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fa8d40a11d0 0x7fa8d40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:00.997 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.016+0000 7fa8f0880700 1 -- 172.21.15.17:0/1736198887 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa8e40cd580 con 0x7fa8ec104ea0 2023-07-14T11:59:00.997 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.016+0000 7fa8eaffd700 1 --2- 172.21.15.17:0/1736198887 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fa8d40a11d0 0x7fa8d40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:00.997 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.017+0000 7fa8eaffd700 1 --2- 172.21.15.17:0/1736198887 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fa8d40a11d0 0x7fa8d40a3680 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc009c60 tx=0x7fa8dc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:01.001 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.020+0000 7fa8f0880700 1 -- 172.21.15.17:0/1736198887 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa8e4097160 con 0x7fa8ec104ea0 2023-07-14T11:59:01.208 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.227+0000 7fa8f1882700 1 -- 172.21.15.17:0/1736198887 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa8ec1a4f70 con 0x7fa8ec104ea0 2023-07-14T11:59:01.209 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.228+0000 7fa8f0880700 1 -- 172.21.15.17:0/1736198887 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+18328 (secure 0 0 0) 0x7fa8e409a780 con 0x7fa8ec104ea0 2023-07-14T11:59:01.210 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:59:01.210 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":80,"fsid":"b3c386cc-223c-11ee-9b33-001a4aab830c","created":"2023-07-14T11:51:59.992960+0000","modified":"2023-07-14T11:58:56.048722+0000","last_up_change":"2023-07-14T11:58:55.037769+0000","last_in_change":"2023-07-14T11:58:37.114870+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:55:28.237002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57003438-a83c-4c8e-a029-56414740895c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6803","nonce":3588366384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6805","nonce":3588366384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6809","nonce":3588366384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6807","nonce":3588366384}]},"public_addr":"172.21.15.17:6803/3588366384","cluster_addr":"172.21.15.17:6805/3588366384","heartbeat_back_addr":"172.21.15.17:6809/3588366384","heartbeat_front_addr":"172.21.15.17:6807/3588366384","state":["exists","up"]},{"osd":1,"uuid":"f56b6337-d8a9-4411-b9d8-b6863dc98556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6801","nonce":3147203064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6803","nonce":3147203064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6807","nonce":3147203064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6805","nonce":3147203064}]},"public_addr":"172.21.15.93:6801/3147203064","cluster_addr":"172.21.15.93:6803/3147203064","heartbeat_back_addr":"172.21.15.93:6807/3147203064","heartbeat_front_addr":"172.21.15.93:6805/3147203064","state":["exists","up"]},{"osd":2,"uuid":"fc34bb7f-551f-46e1-99c0-59924d6c1346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6801","nonce":1877269815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6803","nonce":1877269815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6807","nonce":1877269815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6805","nonce":1877269815}]},"public_addr":"172.21.15.97:6801/1877269815","cluster_addr":"172.21.15.97:6803/1877269815","heartbeat_back_addr":"172.21.15.97:6807/1877269815","heartbeat_front_addr":"172.21.15.97:6805/1877269815","state":["exists","up"]},{"osd":3,"uuid":"37a1583c-4fb8-412a-a0c8-f83b1ff26a91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6811","nonce":2784619821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6813","nonce":2784619821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6817","nonce":2784619821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6815","nonce":2784619821}]},"public_addr":"172.21.15.17:6811/2784619821","cluster_addr":"172.21.15.17:6813/2784619821","heartbeat_back_addr":"172.21.15.17:6817/2784619821","heartbeat_front_addr":"172.21.15.17:6815/2784619821","state":["exists","up"]},{"osd":4,"uuid":"f4b9f377-d519-4b14-ac4a-f1debaeed9f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6809","nonce":3151801173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6811","nonce":3151801173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6815","nonce":3151801173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6813","nonce":3151801173}]},"public_addr":"172.21.15.93:6809/3151801173","cluster_addr":"172.21.15.93:6811/3151801173","heartbeat_back_addr":"172.21.15.93:6815/3151801173","heartbeat_front_addr":"172.21.15.93:6813/3151801173","state":["exists","up"]},{"osd":5,"uuid":"2d2af354-0535-4ef9-b166-3ef219498db8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6809","nonce":2184413924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6811","nonce":2184413924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6815","nonce":2184413924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6813","nonce":2184413924}]},"public_addr":"172.21.15.97:6809/2184413924","cluster_addr":"172.21.15.97:6811/2184413924","heartbeat_back_addr":"172.21.15.97:6815/2184413924","heartbeat_front_addr":"172.21.15.97:6813/2184413924","state":["exists","up"]},{"osd":6,"uuid":"9cdf0c4b-62dc-4168-864e-a9d1650a9ea8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6819","nonce":50486581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6821","nonce":50486581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6825","nonce":50486581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6823","nonce":50486581}]},"public_addr":"172.21.15.17:6819/50486581","cluster_addr":"172.21.15.17:6821/50486581","heartbeat_back_addr":"172.21.15.17:6825/50486581","heartbeat_front_addr":"172.21.15.17:6823/50486581","state":["exists","up"]},{"osd":7,"uuid":"09f60823-5491-45cd-9324-6ae0d56ce262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6817","nonce":2596912392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6819","nonce":2596912392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6823","nonce":2596912392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6821","nonce":2596912392}]},"public_addr":"172.21.15.93:6817/2596912392","cluster_addr":"172.21.15.93:6819/2596912392","heartbeat_back_addr":"172.21.15.93:6823/2596912392","heartbeat_front_addr":"172.21.15.93:6821/2596912392","state":["exists","up"]},{"osd":8,"uuid":"8c5771c9-718e-4313-a2b6-5e1c37c53f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6817","nonce":1965492859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6819","nonce":1965492859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6823","nonce":1965492859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6821","nonce":1965492859}]},"public_addr":"172.21.15.97:6817/1965492859","cluster_addr":"172.21.15.97:6819/1965492859","heartbeat_back_addr":"172.21.15.97:6823/1965492859","heartbeat_front_addr":"172.21.15.97:6821/1965492859","state":["exists","up"]},{"osd":9,"uuid":"1197b8a3-c09f-4c57-aba1-9e988a1620b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6827","nonce":2512645700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6829","nonce":2512645700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6833","nonce":2512645700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6831","nonce":2512645700}]},"public_addr":"172.21.15.17:6827/2512645700","cluster_addr":"172.21.15.17:6829/2512645700","heartbeat_back_addr":"172.21.15.17:6833/2512645700","heartbeat_front_addr":"172.21.15.17:6831/2512645700","state":["exists","up"]},{"osd":10,"uuid":"f99918c2-cdd8-4a7a-be2e-d1d6e0f828a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6825","nonce":264429558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6827","nonce":264429558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6831","nonce":264429558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6829","nonce":264429558}]},"public_addr":"172.21.15.93:6825/264429558","cluster_addr":"172.21.15.93:6827/264429558","heartbeat_back_addr":"172.21.15.93:6831/264429558","heartbeat_front_addr":"172.21.15.93:6829/264429558","state":["exists","up"]},{"osd":11,"uuid":"6bbb5b79-8d4d-441f-9583-203df6c78cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6825","nonce":3389913754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6827","nonce":3389913754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6831","nonce":3389913754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6829","nonce":3389913754}]},"public_addr":"172.21.15.97:6825/3389913754","cluster_addr":"172.21.15.97:6827/3389913754","heartbeat_back_addr":"172.21.15.97:6831/3389913754","heartbeat_front_addr":"172.21.15.97:6829/3389913754","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:54:41.079618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:03.207899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:24.136961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:50.392609+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:56:12.136237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:56:33.212314+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:00.243646+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:22.059234+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:43.232399+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:10.043896+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:31.910780+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:53.158983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/2084033276":"2023-07-15T11:53:05.639597+0000","172.21.15.17:6801/1059284716":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/3651785636":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/1373538024":"2023-07-15T11:52:37.887946+0000","172.21.15.17:0/3560395020":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6800/1059284716":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/4262912861":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/2258375202":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6800/2804227742":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6801/2804227742":"2023-07-15T11:52:37.887946+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:59:01.212 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.230+0000 7fa8f1882700 1 -- 172.21.15.17:0/1736198887 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fa8d40a11d0 msgr2=0x7fa8d40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:01.212 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.230+0000 7fa8f1882700 1 --2- 172.21.15.17:0/1736198887 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fa8d40a11d0 0x7fa8d40a3680 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc009c60 tx=0x7fa8dc009260 comp rx=0 tx=0).stop 2023-07-14T11:59:01.212 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.230+0000 7fa8f1882700 1 -- 172.21.15.17:0/1736198887 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8ec104ea0 msgr2=0x7fa8ec0732d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:01.212 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.230+0000 7fa8f1882700 1 --2- 172.21.15.17:0/1736198887 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8ec104ea0 0x7fa8ec0732d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4007890 tx=0x7fa8e40078c0 comp rx=0 tx=0).stop 2023-07-14T11:59:01.213 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.230+0000 7fa8f1882700 1 -- 172.21.15.17:0/1736198887 shutdown_connections 2023-07-14T11:59:01.213 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.230+0000 7fa8f1882700 1 --2- 172.21.15.17:0/1736198887 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fa8d40a11d0 0x7fa8d40a3680 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:01.213 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.230+0000 7fa8f1882700 1 --2- 172.21.15.17:0/1736198887 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8ec104ea0 0x7fa8ec0732d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:01.213 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.230+0000 7fa8f1882700 1 --2- 172.21.15.17:0/1736198887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8ec0ffe00 0x7fa8ec072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:01.213 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.230+0000 7fa8f1882700 1 --2- 172.21.15.17:0/1736198887 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa8ec1007a0 0x7fa8ec072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:01.213 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.230+0000 7fa8f1882700 1 -- 172.21.15.17:0/1736198887 >> 172.21.15.17:0/1736198887 conn(0x7fa8ec0fb660 msgr2=0x7fa8ec0fc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:01.213 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.230+0000 7fa8f1882700 1 -- 172.21.15.17:0/1736198887 shutdown_connections 2023-07-14T11:59:01.214 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:01.230+0000 7fa8f1882700 1 -- 172.21.15.17:0/1736198887 wait complete. 2023-07-14T11:59:01.611 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:01 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/1736198887' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:59:01.635 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:01 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/1736198887' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:59:01.636 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:01 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/1736198887' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:59:01.707 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:55:28.237002+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T11:59:01.708 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph osd pool get .mgr pg_num 2023-07-14T11:59:02.440 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T11:59:02.611 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:02 smithi093 ceph-mon[128922]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:02.634 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:02 smithi097 ceph-mon[129313]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:02.644 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:02 smithi017 ceph-mon[126339]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:04.033 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.051+0000 7f7400198700 1 -- 172.21.15.17:0/4134037961 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73f8104ea0 msgr2=0x7f73f810ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:04.033 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.051+0000 7f7400198700 1 --2- 172.21.15.17:0/4134037961 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73f8104ea0 0x7f73f810ca20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f73f000b0d0 tx=0x7f73f000b3e0 comp rx=0 tx=0).stop 2023-07-14T11:59:04.034 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.051+0000 7f7400198700 1 -- 172.21.15.17:0/4134037961 shutdown_connections 2023-07-14T11:59:04.034 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.051+0000 7f7400198700 1 --2- 172.21.15.17:0/4134037961 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f73f80ffe00 0x7f73f81001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:04.034 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.051+0000 7f7400198700 1 --2- 172.21.15.17:0/4134037961 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73f8104ea0 0x7f73f810ca20 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:04.034 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.051+0000 7f7400198700 1 --2- 172.21.15.17:0/4134037961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73f81007a0 0x7f73f8104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:04.035 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.051+0000 7f7400198700 1 -- 172.21.15.17:0/4134037961 >> 172.21.15.17:0/4134037961 conn(0x7f73f80fb660 msgr2=0x7f73f80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:04.035 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.051+0000 7f7400198700 1 -- 172.21.15.17:0/4134037961 shutdown_connections 2023-07-14T11:59:04.035 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.051+0000 7f7400198700 1 -- 172.21.15.17:0/4134037961 wait complete. 2023-07-14T11:59:04.038 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.053+0000 7f7400198700 1 Processor -- start 2023-07-14T11:59:04.038 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.053+0000 7f7400198700 1 -- start start 2023-07-14T11:59:04.039 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.053+0000 7f7400198700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f73f80ffe00 0x7f73f8195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:04.039 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.053+0000 7f7400198700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73f81007a0 0x7f73f81962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:04.039 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.053+0000 7f7400198700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73f8104ea0 0x7f73f819a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:04.039 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.053+0000 7f7400198700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f819ad20 con 0x7f73f81007a0 2023-07-14T11:59:04.040 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.053+0000 7f7400198700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f819ae90 con 0x7f73f80ffe00 2023-07-14T11:59:04.040 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.053+0000 7f7400198700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f819b050 con 0x7f73f8104ea0 2023-07-14T11:59:04.040 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.053+0000 7f73fd733700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73f81007a0 0x7f73f81962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:04.040 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.053+0000 7f73fd733700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73f81007a0 0x7f73f81962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36692/0 (socket says 172.21.15.17:36692) 2023-07-14T11:59:04.041 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.053+0000 7f73fdf34700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f73f80ffe00 0x7f73f8195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:04.041 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.053+0000 7f73fd733700 1 -- 172.21.15.17:0/855166769 learned_addr learned my addr 172.21.15.17:0/855166769 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:59:04.041 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.054+0000 7f73fe735700 1 --2- 172.21.15.17:0/855166769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73f8104ea0 0x7f73f819a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:04.041 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.054+0000 7f73fd733700 1 -- 172.21.15.17:0/855166769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73f8104ea0 msgr2=0x7f73f819a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:04.042 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.054+0000 7f73fd733700 1 --2- 172.21.15.17:0/855166769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73f8104ea0 0x7f73f819a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:04.042 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.054+0000 7f73fd733700 1 -- 172.21.15.17:0/855166769 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f73f80ffe00 msgr2=0x7f73f8195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:04.042 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.054+0000 7f73fd733700 1 --2- 172.21.15.17:0/855166769 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f73f80ffe00 0x7f73f8195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:04.043 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.054+0000 7f73fd733700 1 -- 172.21.15.17:0/855166769 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f0009d00 con 0x7f73f81007a0 2023-07-14T11:59:04.043 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.054+0000 7f73fe735700 1 --2- 172.21.15.17:0/855166769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73f8104ea0 0x7f73f819a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:59:04.043 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.054+0000 7f73fd733700 1 --2- 172.21.15.17:0/855166769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73f81007a0 0x7f73f81962a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f73e800e970 tx=0x7f73e800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:04.043 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.054+0000 7f73eeffd700 1 -- 172.21.15.17:0/855166769 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e8009790 con 0x7f73f81007a0 2023-07-14T11:59:04.044 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.054+0000 7f73eeffd700 1 -- 172.21.15.17:0/855166769 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73e800eec0 con 0x7f73f81007a0 2023-07-14T11:59:04.044 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.054+0000 7f7400198700 1 -- 172.21.15.17:0/855166769 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73f819b330 con 0x7f73f81007a0 2023-07-14T11:59:04.044 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.054+0000 7f73eeffd700 1 -- 172.21.15.17:0/855166769 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e8005590 con 0x7f73f81007a0 2023-07-14T11:59:04.045 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.055+0000 7f7400198700 1 -- 172.21.15.17:0/855166769 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73f81a2b90 con 0x7f73f81007a0 2023-07-14T11:59:04.045 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.056+0000 7f7400198700 1 -- 172.21.15.17:0/855166769 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73f804e210 con 0x7f73f81007a0 2023-07-14T11:59:04.046 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.058+0000 7f73eeffd700 1 -- 172.21.15.17:0/855166769 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f73e800cbd0 con 0x7f73f81007a0 2023-07-14T11:59:04.047 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.059+0000 7f73eeffd700 1 --2- 172.21.15.17:0/855166769 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f73e40a1150 0x7f73e40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:04.047 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.059+0000 7f73eeffd700 1 -- 172.21.15.17:0/855166769 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f73e8014070 con 0x7f73f81007a0 2023-07-14T11:59:04.047 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.060+0000 7f73fdf34700 1 --2- 172.21.15.17:0/855166769 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f73e40a1150 0x7f73e40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:04.047 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.060+0000 7f73fdf34700 1 --2- 172.21.15.17:0/855166769 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f73e40a1150 0x7f73e40a3600 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f73f4005950 tx=0x7f73f400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:04.048 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.064+0000 7f73eeffd700 1 -- 172.21.15.17:0/855166769 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f73e8096490 con 0x7f73f81007a0 2023-07-14T11:59:04.251 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.270+0000 7f7400198700 1 -- 172.21.15.17:0/855166769 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f73f81a2e80 con 0x7f73f81007a0 2023-07-14T11:59:04.251 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.270+0000 7f73eeffd700 1 -- 172.21.15.17:0/855166769 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f73e8004ef0 con 0x7f73f81007a0 2023-07-14T11:59:04.252 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2023-07-14T11:59:04.253 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.272+0000 7f7400198700 1 -- 172.21.15.17:0/855166769 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f73e40a1150 msgr2=0x7f73e40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:04.254 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.272+0000 7f7400198700 1 --2- 172.21.15.17:0/855166769 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f73e40a1150 0x7f73e40a3600 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f73f4005950 tx=0x7f73f400b410 comp rx=0 tx=0).stop 2023-07-14T11:59:04.254 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.272+0000 7f7400198700 1 -- 172.21.15.17:0/855166769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73f81007a0 msgr2=0x7f73f81962a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:04.254 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.272+0000 7f7400198700 1 --2- 172.21.15.17:0/855166769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73f81007a0 0x7f73f81962a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f73e800e970 tx=0x7f73e800ec80 comp rx=0 tx=0).stop 2023-07-14T11:59:04.254 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.272+0000 7f7400198700 1 -- 172.21.15.17:0/855166769 shutdown_connections 2023-07-14T11:59:04.254 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.272+0000 7f7400198700 1 --2- 172.21.15.17:0/855166769 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f73e40a1150 0x7f73e40a3600 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:04.254 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.272+0000 7f7400198700 1 --2- 172.21.15.17:0/855166769 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f73f80ffe00 0x7f73f8195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:04.255 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.272+0000 7f7400198700 1 --2- 172.21.15.17:0/855166769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73f8104ea0 0x7f73f819a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:04.255 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.272+0000 7f7400198700 1 --2- 172.21.15.17:0/855166769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73f81007a0 0x7f73f81962a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:04.255 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.272+0000 7f7400198700 1 -- 172.21.15.17:0/855166769 >> 172.21.15.17:0/855166769 conn(0x7f73f80fb660 msgr2=0x7f73f80fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:04.255 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.273+0000 7f7400198700 1 -- 172.21.15.17:0/855166769 shutdown_connections 2023-07-14T11:59:04.255 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:04.273+0000 7f7400198700 1 -- 172.21.15.17:0/855166769 wait complete. 2023-07-14T11:59:04.644 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:04 smithi017 ceph-mon[126339]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:04.644 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:04 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/855166769' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:59:04.721 INFO:tasks.cephadm:Adding mds.a on smithi017 2023-07-14T11:59:04.721 INFO:tasks.cephadm:Adding mds.d on smithi017 2023-07-14T11:59:04.722 INFO:tasks.cephadm:Adding mds.g on smithi017 2023-07-14T11:59:04.722 INFO:tasks.cephadm:Adding mds.j on smithi017 2023-07-14T11:59:04.722 INFO:tasks.cephadm:Adding mds.b on smithi093 2023-07-14T11:59:04.722 INFO:tasks.cephadm:Adding mds.e on smithi093 2023-07-14T11:59:04.723 INFO:tasks.cephadm:Adding mds.h on smithi093 2023-07-14T11:59:04.723 INFO:tasks.cephadm:Adding mds.k on smithi093 2023-07-14T11:59:04.723 INFO:tasks.cephadm:Adding mds.c on smithi097 2023-07-14T11:59:04.723 INFO:tasks.cephadm:Adding mds.f on smithi097 2023-07-14T11:59:04.723 INFO:tasks.cephadm:Adding mds.i on smithi097 2023-07-14T11:59:04.724 INFO:tasks.cephadm:Adding mds.l on smithi097 2023-07-14T11:59:04.724 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph orch apply mds all '12;smithi017=a;smithi017=d;smithi017=g;smithi017=j;smithi093=b;smithi093=e;smithi093=h;smithi093=k;smithi097=c;smithi097=f;smithi097=i;smithi097=l' 2023-07-14T11:59:04.758 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:04 smithi097 ceph-mon[129313]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:04.758 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:04 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/855166769' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:59:04.861 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:04 smithi093 ceph-mon[128922]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:04.861 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:04 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/855166769' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:59:05.294 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T11:59:06.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.502+0000 7fc545086700 1 -- 172.21.15.97:0/139057655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5400ffc80 msgr2=0x7fc5401000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:06.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.502+0000 7fc545086700 1 --2- 172.21.15.97:0/139057655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5400ffc80 0x7fc5401000d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc530009990 tx=0x7fc530009ca0 comp rx=0 tx=0).stop 2023-07-14T11:59:06.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.502+0000 7fc545086700 1 -- 172.21.15.97:0/139057655 shutdown_connections 2023-07-14T11:59:06.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.502+0000 7fc545086700 1 --2- 172.21.15.97:0/139057655 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc540100610 0x7fc540106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:06.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.502+0000 7fc545086700 1 --2- 172.21.15.97:0/139057655 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5400fea80 0x7fc5400fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:06.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.502+0000 7fc545086700 1 --2- 172.21.15.97:0/139057655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5400ffc80 0x7fc5401000d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:06.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.502+0000 7fc545086700 1 -- 172.21.15.97:0/139057655 >> 172.21.15.97:0/139057655 conn(0x7fc5400fa0b0 msgr2=0x7fc5400fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:06.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.502+0000 7fc545086700 1 -- 172.21.15.97:0/139057655 shutdown_connections 2023-07-14T11:59:06.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.502+0000 7fc545086700 1 -- 172.21.15.97:0/139057655 wait complete. 2023-07-14T11:59:06.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.502+0000 7fc545086700 1 Processor -- start 2023-07-14T11:59:06.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.503+0000 7fc545086700 1 -- start start 2023-07-14T11:59:06.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.503+0000 7fc545086700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5400fea80 0x7fc540068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:06.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.503+0000 7fc545086700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5400ffc80 0x7fc54019a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:06.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.503+0000 7fc545086700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc540100610 0x7fc54019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:06.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.503+0000 7fc545086700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54019d3f0 con 0x7fc5400ffc80 2023-07-14T11:59:06.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.503+0000 7fc545086700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54019d560 con 0x7fc540100610 2023-07-14T11:59:06.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.503+0000 7fc545086700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54019d720 con 0x7fc5400fea80 2023-07-14T11:59:06.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.503+0000 7fc53ed9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5400fea80 0x7fc540068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:06.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.503+0000 7fc53ed9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5400fea80 0x7fc540068b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42684/0 (socket says 172.21.15.97:42684) 2023-07-14T11:59:06.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.503+0000 7fc53ed9d700 1 -- 172.21.15.97:0/3218564947 learned_addr learned my addr 172.21.15.97:0/3218564947 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-07-14T11:59:06.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.503+0000 7fc53e59c700 1 --2- 172.21.15.97:0/3218564947 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5400ffc80 0x7fc54019a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:06.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.503+0000 7fc53f59e700 1 --2- 172.21.15.97:0/3218564947 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc540100610 0x7fc54019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:06.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.503+0000 7fc53ed9d700 1 -- 172.21.15.97:0/3218564947 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc540100610 msgr2=0x7fc54019cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:06.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.503+0000 7fc53ed9d700 1 --2- 172.21.15.97:0/3218564947 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc540100610 0x7fc54019cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:06.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.503+0000 7fc53ed9d700 1 -- 172.21.15.97:0/3218564947 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5400ffc80 msgr2=0x7fc54019a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:06.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.503+0000 7fc53ed9d700 1 --2- 172.21.15.97:0/3218564947 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5400ffc80 0x7fc54019a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:06.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.503+0000 7fc53ed9d700 1 -- 172.21.15.97:0/3218564947 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5300095f0 con 0x7fc5400fea80 2023-07-14T11:59:06.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.503+0000 7fc53f59e700 1 --2- 172.21.15.97:0/3218564947 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc540100610 0x7fc54019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T11:59:06.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.503+0000 7fc53ed9d700 1 --2- 172.21.15.97:0/3218564947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5400fea80 0x7fc540068b50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc53400c970 tx=0x7fc53400cd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:06.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.504+0000 7fc527fff700 1 -- 172.21.15.97:0/3218564947 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc534004220 con 0x7fc5400fea80 2023-07-14T11:59:06.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.504+0000 7fc527fff700 1 -- 172.21.15.97:0/3218564947 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc534007d60 con 0x7fc5400fea80 2023-07-14T11:59:06.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.504+0000 7fc527fff700 1 -- 172.21.15.97:0/3218564947 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc534003d00 con 0x7fc5400fea80 2023-07-14T11:59:06.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.504+0000 7fc545086700 1 -- 172.21.15.97:0/3218564947 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc54019da00 con 0x7fc5400fea80 2023-07-14T11:59:06.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.504+0000 7fc545086700 1 -- 172.21.15.97:0/3218564947 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc54019df00 con 0x7fc5400fea80 2023-07-14T11:59:06.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.505+0000 7fc545086700 1 -- 172.21.15.97:0/3218564947 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc54010a2c0 con 0x7fc5400fea80 2023-07-14T11:59:06.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.506+0000 7fc527fff700 1 -- 172.21.15.97:0/3218564947 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc534004380 con 0x7fc5400fea80 2023-07-14T11:59:06.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.506+0000 7fc527fff700 1 --2- 172.21.15.97:0/3218564947 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fc5280a11d0 0x7fc5280a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:06.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.506+0000 7fc527fff700 1 -- 172.21.15.97:0/3218564947 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc5340cc3a0 con 0x7fc5400fea80 2023-07-14T11:59:06.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.507+0000 7fc53e59c700 1 --2- 172.21.15.97:0/3218564947 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fc5280a11d0 0x7fc5280a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:06.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.507+0000 7fc53e59c700 1 --2- 172.21.15.97:0/3218564947 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fc5280a11d0 0x7fc5280a3680 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc530006010 tx=0x7fc53000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:06.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.510+0000 7fc527fff700 1 -- 172.21.15.97:0/3218564947 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc534095f90 con 0x7fc5400fea80 2023-07-14T11:59:06.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.705+0000 7fc545086700 1 -- 172.21.15.97:0/3218564947 --> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi017=a;smithi017=d;smithi017=g;smithi017=j;smithi093=b;smithi093=e;smithi093=h;smithi093=k;smithi097=c;smithi097=f;smithi097=i;smithi097=l", "target": ["mon-mgr", ""]}) v1 -- 0x7fc540060a50 con 0x7fc5280a11d0 2023-07-14T11:59:06.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.714+0000 7fc527fff700 1 -- 172.21.15.97:0/3218564947 <== mgr.14152 v2:172.21.15.17:6800/4237449858 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fc540060a50 con 0x7fc5280a11d0 2023-07-14T11:59:06.696 INFO:teuthology.orchestra.run.smithi097.stdout:Scheduled mds.all update... 2023-07-14T11:59:06.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.716+0000 7fc545086700 1 -- 172.21.15.97:0/3218564947 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fc5280a11d0 msgr2=0x7fc5280a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:06.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.716+0000 7fc545086700 1 --2- 172.21.15.97:0/3218564947 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fc5280a11d0 0x7fc5280a3680 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc530006010 tx=0x7fc53000b540 comp rx=0 tx=0).stop 2023-07-14T11:59:06.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.716+0000 7fc545086700 1 -- 172.21.15.97:0/3218564947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5400fea80 msgr2=0x7fc540068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:06.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.716+0000 7fc545086700 1 --2- 172.21.15.97:0/3218564947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5400fea80 0x7fc540068b50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc53400c970 tx=0x7fc53400cd30 comp rx=0 tx=0).stop 2023-07-14T11:59:06.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.716+0000 7fc545086700 1 -- 172.21.15.97:0/3218564947 shutdown_connections 2023-07-14T11:59:06.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.716+0000 7fc545086700 1 --2- 172.21.15.97:0/3218564947 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fc5280a11d0 0x7fc5280a3680 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:06.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.716+0000 7fc545086700 1 --2- 172.21.15.97:0/3218564947 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc540100610 0x7fc54019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:06.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.716+0000 7fc545086700 1 --2- 172.21.15.97:0/3218564947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5400fea80 0x7fc540068b50 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:06.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.716+0000 7fc545086700 1 --2- 172.21.15.97:0/3218564947 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5400ffc80 0x7fc54019a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:06.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.716+0000 7fc545086700 1 -- 172.21.15.97:0/3218564947 >> 172.21.15.97:0/3218564947 conn(0x7fc5400fa0b0 msgr2=0x7fc5400fc330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:06.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.716+0000 7fc545086700 1 -- 172.21.15.97:0/3218564947 shutdown_connections 2023-07-14T11:59:06.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-07-14T11:59:06.716+0000 7fc545086700 1 -- 172.21.15.97:0/3218564947 wait complete. 2023-07-14T11:59:07.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:06 smithi093 ceph-mon[128922]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:07.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:06 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:07.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:06 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:59:07.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:06 smithi097 ceph-mon[129313]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:07.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:06 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:07.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:06 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:59:07.141 DEBUG:teuthology.orchestra.run.smithi017:mds.a> sudo journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mds.a.service 2023-07-14T11:59:07.146 DEBUG:teuthology.orchestra.run.smithi017:mds.d> sudo journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mds.d.service 2023-07-14T11:59:07.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:06 smithi017 ceph-mon[126339]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:07.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:06 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:07.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:06 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:59:07.192 DEBUG:teuthology.orchestra.run.smithi017:mds.g> sudo journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mds.g.service 2023-07-14T11:59:07.232 INFO:journalctl@ceph.mds.a.smithi017.stdout:-- Logs begin at Fri 2023-07-14 11:32:03 UTC. -- 2023-07-14T11:59:07.237 DEBUG:teuthology.orchestra.run.smithi017:mds.j> sudo journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mds.j.service 2023-07-14T11:59:07.272 INFO:journalctl@ceph.mds.d.smithi017.stdout:-- Logs begin at Fri 2023-07-14 11:32:03 UTC. -- 2023-07-14T11:59:07.275 DEBUG:teuthology.orchestra.run.smithi093:mds.b> sudo journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mds.b.service 2023-07-14T11:59:07.277 DEBUG:teuthology.orchestra.run.smithi093:mds.e> sudo journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mds.e.service 2023-07-14T11:59:07.307 INFO:journalctl@ceph.mds.b.smithi093.stdout:-- Logs begin at Fri 2023-07-14 11:24:16 UTC. -- 2023-07-14T11:59:07.308 DEBUG:teuthology.orchestra.run.smithi093:mds.h> sudo journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mds.h.service 2023-07-14T11:59:07.312 INFO:journalctl@ceph.mds.g.smithi017.stdout:-- Logs begin at Fri 2023-07-14 11:32:03 UTC. -- 2023-07-14T11:59:07.334 INFO:journalctl@ceph.mds.e.smithi093.stdout:-- Logs begin at Fri 2023-07-14 11:24:16 UTC. -- 2023-07-14T11:59:07.335 DEBUG:teuthology.orchestra.run.smithi093:mds.k> sudo journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mds.k.service 2023-07-14T11:59:07.347 INFO:journalctl@ceph.mds.j.smithi017.stdout:-- Logs begin at Fri 2023-07-14 11:32:03 UTC. -- 2023-07-14T11:59:07.360 INFO:journalctl@ceph.mds.h.smithi093.stdout:-- Logs begin at Fri 2023-07-14 11:24:16 UTC. -- 2023-07-14T11:59:07.361 DEBUG:teuthology.orchestra.run.smithi097:mds.c> sudo journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mds.c.service 2023-07-14T11:59:07.363 DEBUG:teuthology.orchestra.run.smithi097:mds.f> sudo journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mds.f.service 2023-07-14T11:59:07.386 INFO:journalctl@ceph.mds.k.smithi093.stdout:-- Logs begin at Fri 2023-07-14 11:24:16 UTC. -- 2023-07-14T11:59:07.392 INFO:journalctl@ceph.mds.c.smithi097.stdout:-- Logs begin at Fri 2023-07-14 11:24:16 UTC. -- 2023-07-14T11:59:07.393 DEBUG:teuthology.orchestra.run.smithi097:mds.i> sudo journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mds.i.service 2023-07-14T11:59:07.419 INFO:journalctl@ceph.mds.f.smithi097.stdout:-- Logs begin at Fri 2023-07-14 11:24:16 UTC. -- 2023-07-14T11:59:07.420 DEBUG:teuthology.orchestra.run.smithi097:mds.l> sudo journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mds.l.service 2023-07-14T11:59:07.446 INFO:journalctl@ceph.mds.i.smithi097.stdout:-- Logs begin at Fri 2023-07-14 11:24:16 UTC. -- 2023-07-14T11:59:07.446 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-07-14T11:59:07.447 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:59:07.472 INFO:journalctl@ceph.mds.l.smithi097.stdout:-- Logs begin at Fri 2023-07-14 11:24:16 UTC. -- 2023-07-14T11:59:07.669 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.688+0000 7f96f6fff700 1 -- 172.21.15.17:0/858647281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96f01017e0 msgr2=0x7f96f0108080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:07.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.688+0000 7f96f6fff700 1 --2- 172.21.15.17:0/858647281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96f01017e0 0x7f96f0108080 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f96e800b0a0 tx=0x7f96e800b3b0 comp rx=0 tx=0).stop 2023-07-14T11:59:07.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.688+0000 7f96f6fff700 1 -- 172.21.15.17:0/858647281 shutdown_connections 2023-07-14T11:59:07.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.688+0000 7f96f6fff700 1 --2- 172.21.15.17:0/858647281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96f01017e0 0x7f96f0108080 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:07.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.688+0000 7f96f6fff700 1 --2- 172.21.15.17:0/858647281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96f00ffba0 0x7f96f00fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:07.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.688+0000 7f96f6fff700 1 --2- 172.21.15.17:0/858647281 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f96f0100e30 0x7f96f01012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:07.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.688+0000 7f96f6fff700 1 -- 172.21.15.17:0/858647281 >> 172.21.15.17:0/858647281 conn(0x7f96f00fb1c0 msgr2=0x7f96f00fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:07.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.688+0000 7f96f6fff700 1 -- 172.21.15.17:0/858647281 shutdown_connections 2023-07-14T11:59:07.671 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.688+0000 7f96f6fff700 1 -- 172.21.15.17:0/858647281 wait complete. 2023-07-14T11:59:07.672 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.689+0000 7f96f6fff700 1 Processor -- start 2023-07-14T11:59:07.673 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.689+0000 7f96f6fff700 1 -- start start 2023-07-14T11:59:07.673 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.690+0000 7f96f6fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96f00ffba0 0x7f96f00742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:07.673 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.690+0000 7f96f6fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96f0100e30 0x7f96f0072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:07.673 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.690+0000 7f96f6fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f96f01017e0 0x7f96f0072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:07.673 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.690+0000 7f96f6fff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f00735e0 con 0x7f96f0100e30 2023-07-14T11:59:07.673 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.690+0000 7f96f6fff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f0073750 con 0x7f96f01017e0 2023-07-14T11:59:07.673 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.690+0000 7f96f6fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f019fad0 con 0x7f96f00ffba0 2023-07-14T11:59:07.674 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.690+0000 7f96f4d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96f00ffba0 0x7f96f00742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:07.674 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.690+0000 7f96f4d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96f00ffba0 0x7f96f00742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.17:47814/0 (socket says 172.21.15.17:47814) 2023-07-14T11:59:07.674 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.690+0000 7f96f4d9b700 1 -- 172.21.15.17:0/3985388236 learned_addr learned my addr 172.21.15.17:0/3985388236 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:59:07.674 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.690+0000 7f96effff700 1 --2- 172.21.15.17:0/3985388236 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96f0100e30 0x7f96f0072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:07.674 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.690+0000 7f96f559c700 1 --2- 172.21.15.17:0/3985388236 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f96f01017e0 0x7f96f0072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:07.674 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.690+0000 7f96f4d9b700 1 -- 172.21.15.17:0/3985388236 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f96f01017e0 msgr2=0x7f96f0072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:07.675 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.690+0000 7f96f4d9b700 1 --2- 172.21.15.17:0/3985388236 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f96f01017e0 0x7f96f0072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:07.675 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.690+0000 7f96f4d9b700 1 -- 172.21.15.17:0/3985388236 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96f0100e30 msgr2=0x7f96f0072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:07.675 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.690+0000 7f96f4d9b700 1 --2- 172.21.15.17:0/3985388236 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96f0100e30 0x7f96f0072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:07.675 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.690+0000 7f96f4d9b700 1 -- 172.21.15.17:0/3985388236 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e8009d00 con 0x7f96f00ffba0 2023-07-14T11:59:07.675 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.690+0000 7f96effff700 1 --2- 172.21.15.17:0/3985388236 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96f0100e30 0x7f96f0072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:59:07.675 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.691+0000 7f96f4d9b700 1 --2- 172.21.15.17:0/3985388236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96f00ffba0 0x7f96f00742c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f96e400e9a0 tx=0x7f96e400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:07.676 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.691+0000 7f96edffb700 1 -- 172.21.15.17:0/3985388236 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e4009790 con 0x7f96f00ffba0 2023-07-14T11:59:07.676 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.691+0000 7f96edffb700 1 -- 172.21.15.17:0/3985388236 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96e400cbd0 con 0x7f96f00ffba0 2023-07-14T11:59:07.676 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.691+0000 7f96edffb700 1 -- 172.21.15.17:0/3985388236 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e4018680 con 0x7f96f00ffba0 2023-07-14T11:59:07.676 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.691+0000 7f96f6fff700 1 -- 172.21.15.17:0/3985388236 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96f019fdb0 con 0x7f96f00ffba0 2023-07-14T11:59:07.676 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.691+0000 7f96f6fff700 1 -- 172.21.15.17:0/3985388236 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96f01a0180 con 0x7f96f00ffba0 2023-07-14T11:59:07.678 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.692+0000 7f96f6fff700 1 -- 172.21.15.17:0/3985388236 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96f0107190 con 0x7f96f00ffba0 2023-07-14T11:59:07.678 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.694+0000 7f96edffb700 1 -- 172.21.15.17:0/3985388236 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f96e400cd40 con 0x7f96f00ffba0 2023-07-14T11:59:07.678 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.695+0000 7f96edffb700 1 --2- 172.21.15.17:0/3985388236 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f96e00a11d0 0x7f96e00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:07.679 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.695+0000 7f96edffb700 1 -- 172.21.15.17:0/3985388236 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f96e4014070 con 0x7f96f00ffba0 2023-07-14T11:59:07.679 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.696+0000 7f96effff700 1 --2- 172.21.15.17:0/3985388236 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f96e00a11d0 0x7f96e00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:07.679 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.696+0000 7f96effff700 1 --2- 172.21.15.17:0/3985388236 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f96e00a11d0 0x7f96e00a3680 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f96dc00bb90 tx=0x7f96dc005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:07.680 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.699+0000 7f96edffb700 1 -- 172.21.15.17:0/3985388236 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f96e4051790 con 0x7f96f00ffba0 2023-07-14T11:59:07.894 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.913+0000 7f96f6fff700 1 -- 172.21.15.17:0/3985388236 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f96f002cab0 con 0x7f96f00ffba0 2023-07-14T11:59:07.894 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:07 smithi017 ceph-mon[126339]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi017=a;smithi017=d;smithi017=g;smithi017=j;smithi093=b;smithi093=e;smithi093=h;smithi093=k;smithi097=c;smithi097=f;smithi097=i;smithi097=l", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:59:07.894 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:07 smithi017 ceph-mon[126339]: Saving service mds.all spec with placement smithi017=a;smithi017=d;smithi017=g;smithi017=j;smithi093=b;smithi093=e;smithi093=h;smithi093=k;smithi097=c;smithi097=f;smithi097=i;smithi097=l;count:12 2023-07-14T11:59:07.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:07 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:59:07.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:07 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:59:07.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:07 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:07.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:07 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:59:07.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:07 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:59:07.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:07 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:59:07.897 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.914+0000 7f96edffb700 1 -- 172.21.15.17:0/3985388236 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+18328 (secure 0 0 0) 0x7f96e4096280 con 0x7f96f00ffba0 2023-07-14T11:59:07.898 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:59:07.898 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":80,"fsid":"b3c386cc-223c-11ee-9b33-001a4aab830c","created":"2023-07-14T11:51:59.992960+0000","modified":"2023-07-14T11:58:56.048722+0000","last_up_change":"2023-07-14T11:58:55.037769+0000","last_in_change":"2023-07-14T11:58:37.114870+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:55:28.237002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57003438-a83c-4c8e-a029-56414740895c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6803","nonce":3588366384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6805","nonce":3588366384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6809","nonce":3588366384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6807","nonce":3588366384}]},"public_addr":"172.21.15.17:6803/3588366384","cluster_addr":"172.21.15.17:6805/3588366384","heartbeat_back_addr":"172.21.15.17:6809/3588366384","heartbeat_front_addr":"172.21.15.17:6807/3588366384","state":["exists","up"]},{"osd":1,"uuid":"f56b6337-d8a9-4411-b9d8-b6863dc98556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6801","nonce":3147203064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6803","nonce":3147203064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6807","nonce":3147203064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6805","nonce":3147203064}]},"public_addr":"172.21.15.93:6801/3147203064","cluster_addr":"172.21.15.93:6803/3147203064","heartbeat_back_addr":"172.21.15.93:6807/3147203064","heartbeat_front_addr":"172.21.15.93:6805/3147203064","state":["exists","up"]},{"osd":2,"uuid":"fc34bb7f-551f-46e1-99c0-59924d6c1346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6801","nonce":1877269815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6803","nonce":1877269815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6807","nonce":1877269815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6805","nonce":1877269815}]},"public_addr":"172.21.15.97:6801/1877269815","cluster_addr":"172.21.15.97:6803/1877269815","heartbeat_back_addr":"172.21.15.97:6807/1877269815","heartbeat_front_addr":"172.21.15.97:6805/1877269815","state":["exists","up"]},{"osd":3,"uuid":"37a1583c-4fb8-412a-a0c8-f83b1ff26a91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6811","nonce":2784619821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6813","nonce":2784619821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6817","nonce":2784619821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6815","nonce":2784619821}]},"public_addr":"172.21.15.17:6811/2784619821","cluster_addr":"172.21.15.17:6813/2784619821","heartbeat_back_addr":"172.21.15.17:6817/2784619821","heartbeat_front_addr":"172.21.15.17:6815/2784619821","state":["exists","up"]},{"osd":4,"uuid":"f4b9f377-d519-4b14-ac4a-f1debaeed9f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6809","nonce":3151801173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6811","nonce":3151801173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6815","nonce":3151801173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6813","nonce":3151801173}]},"public_addr":"172.21.15.93:6809/3151801173","cluster_addr":"172.21.15.93:6811/3151801173","heartbeat_back_addr":"172.21.15.93:6815/3151801173","heartbeat_front_addr":"172.21.15.93:6813/3151801173","state":["exists","up"]},{"osd":5,"uuid":"2d2af354-0535-4ef9-b166-3ef219498db8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6809","nonce":2184413924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6811","nonce":2184413924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6815","nonce":2184413924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6813","nonce":2184413924}]},"public_addr":"172.21.15.97:6809/2184413924","cluster_addr":"172.21.15.97:6811/2184413924","heartbeat_back_addr":"172.21.15.97:6815/2184413924","heartbeat_front_addr":"172.21.15.97:6813/2184413924","state":["exists","up"]},{"osd":6,"uuid":"9cdf0c4b-62dc-4168-864e-a9d1650a9ea8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6819","nonce":50486581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6821","nonce":50486581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6825","nonce":50486581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6823","nonce":50486581}]},"public_addr":"172.21.15.17:6819/50486581","cluster_addr":"172.21.15.17:6821/50486581","heartbeat_back_addr":"172.21.15.17:6825/50486581","heartbeat_front_addr":"172.21.15.17:6823/50486581","state":["exists","up"]},{"osd":7,"uuid":"09f60823-5491-45cd-9324-6ae0d56ce262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6817","nonce":2596912392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6819","nonce":2596912392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6823","nonce":2596912392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6821","nonce":2596912392}]},"public_addr":"172.21.15.93:6817/2596912392","cluster_addr":"172.21.15.93:6819/2596912392","heartbeat_back_addr":"172.21.15.93:6823/2596912392","heartbeat_front_addr":"172.21.15.93:6821/2596912392","state":["exists","up"]},{"osd":8,"uuid":"8c5771c9-718e-4313-a2b6-5e1c37c53f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6817","nonce":1965492859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6819","nonce":1965492859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6823","nonce":1965492859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6821","nonce":1965492859}]},"public_addr":"172.21.15.97:6817/1965492859","cluster_addr":"172.21.15.97:6819/1965492859","heartbeat_back_addr":"172.21.15.97:6823/1965492859","heartbeat_front_addr":"172.21.15.97:6821/1965492859","state":["exists","up"]},{"osd":9,"uuid":"1197b8a3-c09f-4c57-aba1-9e988a1620b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6827","nonce":2512645700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6829","nonce":2512645700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6833","nonce":2512645700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6831","nonce":2512645700}]},"public_addr":"172.21.15.17:6827/2512645700","cluster_addr":"172.21.15.17:6829/2512645700","heartbeat_back_addr":"172.21.15.17:6833/2512645700","heartbeat_front_addr":"172.21.15.17:6831/2512645700","state":["exists","up"]},{"osd":10,"uuid":"f99918c2-cdd8-4a7a-be2e-d1d6e0f828a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6825","nonce":264429558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6827","nonce":264429558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6831","nonce":264429558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6829","nonce":264429558}]},"public_addr":"172.21.15.93:6825/264429558","cluster_addr":"172.21.15.93:6827/264429558","heartbeat_back_addr":"172.21.15.93:6831/264429558","heartbeat_front_addr":"172.21.15.93:6829/264429558","state":["exists","up"]},{"osd":11,"uuid":"6bbb5b79-8d4d-441f-9583-203df6c78cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6825","nonce":3389913754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6827","nonce":3389913754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6831","nonce":3389913754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6829","nonce":3389913754}]},"public_addr":"172.21.15.97:6825/3389913754","cluster_addr":"172.21.15.97:6827/3389913754","heartbeat_back_addr":"172.21.15.97:6831/3389913754","heartbeat_front_addr":"172.21.15.97:6829/3389913754","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:54:41.079618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:03.207899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:24.136961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:50.392609+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:56:12.136237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:56:33.212314+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:00.243646+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:22.059234+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:43.232399+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:10.043896+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:31.910780+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:53.158983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/2084033276":"2023-07-15T11:53:05.639597+0000","172.21.15.17:6801/1059284716":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/3651785636":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/1373538024":"2023-07-15T11:52:37.887946+0000","172.21.15.17:0/3560395020":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6800/1059284716":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/4262912861":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/2258375202":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6800/2804227742":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6801/2804227742":"2023-07-15T11:52:37.887946+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:59:07.899 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.916+0000 7f96f6fff700 1 -- 172.21.15.17:0/3985388236 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f96e00a11d0 msgr2=0x7f96e00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:07.899 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.916+0000 7f96f6fff700 1 --2- 172.21.15.17:0/3985388236 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f96e00a11d0 0x7f96e00a3680 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f96dc00bb90 tx=0x7f96dc005d80 comp rx=0 tx=0).stop 2023-07-14T11:59:07.899 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.916+0000 7f96f6fff700 1 -- 172.21.15.17:0/3985388236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96f00ffba0 msgr2=0x7f96f00742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:07.899 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.916+0000 7f96f6fff700 1 --2- 172.21.15.17:0/3985388236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96f00ffba0 0x7f96f00742c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f96e400e9a0 tx=0x7f96e400ecb0 comp rx=0 tx=0).stop 2023-07-14T11:59:07.900 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.916+0000 7f96f6fff700 1 -- 172.21.15.17:0/3985388236 shutdown_connections 2023-07-14T11:59:07.900 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.916+0000 7f96f6fff700 1 --2- 172.21.15.17:0/3985388236 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f96e00a11d0 0x7f96e00a3680 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:07.900 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.916+0000 7f96f6fff700 1 --2- 172.21.15.17:0/3985388236 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f96f01017e0 0x7f96f0072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:07.900 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.916+0000 7f96f6fff700 1 --2- 172.21.15.17:0/3985388236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96f00ffba0 0x7f96f00742c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:07.900 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.916+0000 7f96f6fff700 1 --2- 172.21.15.17:0/3985388236 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96f0100e30 0x7f96f0072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:07.900 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.916+0000 7f96f6fff700 1 -- 172.21.15.17:0/3985388236 >> 172.21.15.17:0/3985388236 conn(0x7f96f00fb1c0 msgr2=0x7f96f01042d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:07.901 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.917+0000 7f96f6fff700 1 -- 172.21.15.17:0/3985388236 shutdown_connections 2023-07-14T11:59:07.901 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:07.917+0000 7f96f6fff700 1 -- 172.21.15.17:0/3985388236 wait complete. 2023-07-14T11:59:07.917 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:55:28.237002+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T11:59:07.918 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:59:08.003 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:07 smithi097 ceph-mon[129313]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi017=a;smithi017=d;smithi017=g;smithi017=j;smithi093=b;smithi093=e;smithi093=h;smithi093=k;smithi097=c;smithi097=f;smithi097=i;smithi097=l", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:59:08.003 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:07 smithi097 ceph-mon[129313]: Saving service mds.all spec with placement smithi017=a;smithi017=d;smithi017=g;smithi017=j;smithi093=b;smithi093=e;smithi093=h;smithi093=k;smithi097=c;smithi097=f;smithi097=i;smithi097=l;count:12 2023-07-14T11:59:08.003 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:07 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:59:08.003 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:07 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:59:08.004 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:07 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:08.004 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:07 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:59:08.004 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:07 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:59:08.004 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:07 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:59:08.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:07 smithi093 ceph-mon[128922]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi017=a;smithi017=d;smithi017=g;smithi017=j;smithi093=b;smithi093=e;smithi093=h;smithi093=k;smithi097=c;smithi097=f;smithi097=i;smithi097=l", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T11:59:08.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:07 smithi093 ceph-mon[128922]: Saving service mds.all spec with placement smithi017=a;smithi017=d;smithi017=g;smithi017=j;smithi093=b;smithi093=e;smithi093=h;smithi093=k;smithi097=c;smithi097=f;smithi097=i;smithi097=l;count:12 2023-07-14T11:59:08.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:07 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:59:08.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:07 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:59:08.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:07 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:08.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:07 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:59:08.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:07 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:59:08.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:07 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:59:08.124 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.143+0000 7f344b9ff700 1 -- 172.21.15.17:0/4115218620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34440ffbe0 msgr2=0x7f34440ffff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:08.125 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.143+0000 7f344b9ff700 1 --2- 172.21.15.17:0/4115218620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34440ffbe0 0x7f34440ffff0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3440009990 tx=0x7f3440009ca0 comp rx=0 tx=0).stop 2023-07-14T11:59:08.125 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.143+0000 7f344b9ff700 1 -- 172.21.15.17:0/4115218620 shutdown_connections 2023-07-14T11:59:08.125 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.143+0000 7f344b9ff700 1 --2- 172.21.15.17:0/4115218620 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3444101760 0x7f3444108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:08.125 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.143+0000 7f344b9ff700 1 --2- 172.21.15.17:0/4115218620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34440ffbe0 0x7f34440ffff0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:08.125 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.143+0000 7f344b9ff700 1 --2- 172.21.15.17:0/4115218620 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3444100dd0 0x7f3444101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:08.125 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.143+0000 7f344b9ff700 1 -- 172.21.15.17:0/4115218620 >> 172.21.15.17:0/4115218620 conn(0x7f34440fb200 msgr2=0x7f34440fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:08.126 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.144+0000 7f344b9ff700 1 -- 172.21.15.17:0/4115218620 shutdown_connections 2023-07-14T11:59:08.126 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.144+0000 7f344b9ff700 1 -- 172.21.15.17:0/4115218620 wait complete. 2023-07-14T11:59:08.127 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.144+0000 7f344b9ff700 1 Processor -- start 2023-07-14T11:59:08.128 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.145+0000 7f344b9ff700 1 -- start start 2023-07-14T11:59:08.128 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.145+0000 7f344b9ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34440ffbe0 0x7f3444197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:08.128 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.145+0000 7f344b9ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3444100dd0 0x7f3444198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:08.128 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.145+0000 7f344b9ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3444101760 0x7f344419f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:08.128 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.145+0000 7f344b9ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344419f680 con 0x7f3444100dd0 2023-07-14T11:59:08.128 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.145+0000 7f344b9ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344419f7c0 con 0x7f3444101760 2023-07-14T11:59:08.129 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.145+0000 7f344b9ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344419f950 con 0x7f34440ffbe0 2023-07-14T11:59:08.129 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.145+0000 7f3448f9a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3444100dd0 0x7f3444198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:08.129 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.145+0000 7f344979b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34440ffbe0 0x7f3444197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:08.129 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.145+0000 7f344979b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34440ffbe0 0x7f3444197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.17:47844/0 (socket says 172.21.15.17:47844) 2023-07-14T11:59:08.129 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.145+0000 7f344979b700 1 -- 172.21.15.17:0/479441687 learned_addr learned my addr 172.21.15.17:0/479441687 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:59:08.129 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.145+0000 7f3449f9c700 1 --2- 172.21.15.17:0/479441687 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3444101760 0x7f344419f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:08.129 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.146+0000 7f344979b700 1 -- 172.21.15.17:0/479441687 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3444101760 msgr2=0x7f344419f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:08.129 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.146+0000 7f344979b700 1 --2- 172.21.15.17:0/479441687 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3444101760 0x7f344419f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:08.130 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.146+0000 7f344979b700 1 -- 172.21.15.17:0/479441687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3444100dd0 msgr2=0x7f3444198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:08.130 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.146+0000 7f344979b700 1 --2- 172.21.15.17:0/479441687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3444100dd0 0x7f3444198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:08.130 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.146+0000 7f344979b700 1 -- 172.21.15.17:0/479441687 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34400095f0 con 0x7f34440ffbe0 2023-07-14T11:59:08.130 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.146+0000 7f344979b700 1 --2- 172.21.15.17:0/479441687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34440ffbe0 0x7f3444197b00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3440004710 tx=0x7f3440009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:08.130 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.146+0000 7f343a7fc700 1 -- 172.21.15.17:0/479441687 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344001d070 con 0x7f34440ffbe0 2023-07-14T11:59:08.130 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.146+0000 7f344b9ff700 1 -- 172.21.15.17:0/479441687 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f344419fbd0 con 0x7f34440ffbe0 2023-07-14T11:59:08.131 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.146+0000 7f343a7fc700 1 -- 172.21.15.17:0/479441687 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3440021410 con 0x7f34440ffbe0 2023-07-14T11:59:08.131 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.146+0000 7f343a7fc700 1 -- 172.21.15.17:0/479441687 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3440017650 con 0x7f34440ffbe0 2023-07-14T11:59:08.131 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.146+0000 7f344b9ff700 1 -- 172.21.15.17:0/479441687 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34441a00a0 con 0x7f34440ffbe0 2023-07-14T11:59:08.132 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.147+0000 7f344b9ff700 1 -- 172.21.15.17:0/479441687 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3444192260 con 0x7f34440ffbe0 2023-07-14T11:59:08.132 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.149+0000 7f343a7fc700 1 -- 172.21.15.17:0/479441687 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3440021580 con 0x7f34440ffbe0 2023-07-14T11:59:08.132 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.151+0000 7f343a7fc700 1 --2- 172.21.15.17:0/479441687 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f34300a1150 0x7f34300a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:08.132 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.151+0000 7f343a7fc700 1 -- 172.21.15.17:0/479441687 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f34400cd820 con 0x7f34440ffbe0 2023-07-14T11:59:08.132 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.151+0000 7f3448f9a700 1 --2- 172.21.15.17:0/479441687 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f34300a1150 0x7f34300a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:08.133 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.151+0000 7f3448f9a700 1 --2- 172.21.15.17:0/479441687 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f34300a1150 0x7f34300a3600 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3434009c60 tx=0x7f3434009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:08.133 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.152+0000 7f343a7fc700 1 -- 172.21.15.17:0/479441687 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3440097400 con 0x7f34440ffbe0 2023-07-14T11:59:08.339 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.358+0000 7f344b9ff700 1 -- 172.21.15.17:0/479441687 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f344402c850 con 0x7f34440ffbe0 2023-07-14T11:59:08.340 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.359+0000 7f343a7fc700 1 -- 172.21.15.17:0/479441687 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f344009aa20 con 0x7f34440ffbe0 2023-07-14T11:59:08.340 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2023-07-14T11:59:08.343 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.362+0000 7f344b9ff700 1 -- 172.21.15.17:0/479441687 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f34300a1150 msgr2=0x7f34300a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:08.343 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.362+0000 7f344b9ff700 1 --2- 172.21.15.17:0/479441687 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f34300a1150 0x7f34300a3600 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3434009c60 tx=0x7f3434009260 comp rx=0 tx=0).stop 2023-07-14T11:59:08.343 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.362+0000 7f344b9ff700 1 -- 172.21.15.17:0/479441687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34440ffbe0 msgr2=0x7f3444197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:08.343 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.362+0000 7f344b9ff700 1 --2- 172.21.15.17:0/479441687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34440ffbe0 0x7f3444197b00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3440004710 tx=0x7f3440009580 comp rx=0 tx=0).stop 2023-07-14T11:59:08.343 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.362+0000 7f344b9ff700 1 -- 172.21.15.17:0/479441687 shutdown_connections 2023-07-14T11:59:08.343 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.362+0000 7f344b9ff700 1 --2- 172.21.15.17:0/479441687 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f34300a1150 0x7f34300a3600 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:08.344 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.362+0000 7f344b9ff700 1 --2- 172.21.15.17:0/479441687 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3444101760 0x7f344419f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:08.344 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.362+0000 7f344b9ff700 1 --2- 172.21.15.17:0/479441687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34440ffbe0 0x7f3444197b00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:08.344 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.362+0000 7f344b9ff700 1 --2- 172.21.15.17:0/479441687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3444100dd0 0x7f3444198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:08.344 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.362+0000 7f344b9ff700 1 -- 172.21.15.17:0/479441687 >> 172.21.15.17:0/479441687 conn(0x7f34440fb200 msgr2=0x7f34441066d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:08.344 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.362+0000 7f344b9ff700 1 -- 172.21.15.17:0/479441687 shutdown_connections 2023-07-14T11:59:08.344 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.362+0000 7f344b9ff700 1 -- 172.21.15.17:0/479441687 wait complete. 2023-07-14T11:59:08.362 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:59:08.565 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.584+0000 7f6feb5ff700 1 -- 172.21.15.17:0/692147219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fe4107a20 msgr2=0x7f6fe4109e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:08.566 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.584+0000 7f6feb5ff700 1 --2- 172.21.15.17:0/692147219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fe4107a20 0x7f6fe4109e40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc009f90 tx=0x7f6fdc00b330 comp rx=0 tx=0).stop 2023-07-14T11:59:08.566 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.584+0000 7f6feb5ff700 1 -- 172.21.15.17:0/692147219 shutdown_connections 2023-07-14T11:59:08.566 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.584+0000 7f6feb5ff700 1 --2- 172.21.15.17:0/692147219 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6fe40744b0 0x7f6fe40729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:08.566 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.584+0000 7f6feb5ff700 1 --2- 172.21.15.17:0/692147219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fe4107a20 0x7f6fe4109e40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:08.566 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.584+0000 7f6feb5ff700 1 --2- 172.21.15.17:0/692147219 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fe4072ee0 0x7f6fe4073350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:08.566 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.584+0000 7f6feb5ff700 1 -- 172.21.15.17:0/692147219 >> 172.21.15.17:0/692147219 conn(0x7f6fe40fb1c0 msgr2=0x7f6fe40fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:08.567 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.584+0000 7f6feb5ff700 1 -- 172.21.15.17:0/692147219 shutdown_connections 2023-07-14T11:59:08.567 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.584+0000 7f6feb5ff700 1 -- 172.21.15.17:0/692147219 wait complete. 2023-07-14T11:59:08.568 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.586+0000 7f6feb5ff700 1 Processor -- start 2023-07-14T11:59:08.569 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.586+0000 7f6feb5ff700 1 -- start start 2023-07-14T11:59:08.569 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.586+0000 7f6feb5ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fe4072ee0 0x7f6fe4197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:08.569 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.586+0000 7f6feb5ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fe40744b0 0x7f6fe4198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:08.570 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.587+0000 7f6feb5ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6fe4107a20 0x7f6fe419f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:08.570 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.587+0000 7f6feb5ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe419f790 con 0x7f6fe4072ee0 2023-07-14T11:59:08.570 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.587+0000 7f6feb5ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe419f900 con 0x7f6fe4107a20 2023-07-14T11:59:08.571 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.587+0000 7f6feb5ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe419fac0 con 0x7f6fe40744b0 2023-07-14T11:59:08.571 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.587+0000 7f6fe939b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fe4072ee0 0x7f6fe4197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:08.571 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.587+0000 7f6fe939b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fe4072ee0 0x7f6fe4197c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36768/0 (socket says 172.21.15.17:36768) 2023-07-14T11:59:08.571 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.587+0000 7f6fe939b700 1 -- 172.21.15.17:0/4048563501 learned_addr learned my addr 172.21.15.17:0/4048563501 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:59:08.572 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.587+0000 7f6fe8b9a700 1 --2- 172.21.15.17:0/4048563501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fe40744b0 0x7f6fe4198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:08.572 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.587+0000 7f6fe9b9c700 1 --2- 172.21.15.17:0/4048563501 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6fe4107a20 0x7f6fe419f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:08.572 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.587+0000 7f6fe939b700 1 -- 172.21.15.17:0/4048563501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fe40744b0 msgr2=0x7f6fe4198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:08.573 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.587+0000 7f6fe939b700 1 --2- 172.21.15.17:0/4048563501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fe40744b0 0x7f6fe4198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:08.573 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.587+0000 7f6fe939b700 1 -- 172.21.15.17:0/4048563501 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6fe4107a20 msgr2=0x7f6fe419f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:08.573 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.587+0000 7f6fe939b700 1 --2- 172.21.15.17:0/4048563501 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6fe4107a20 0x7f6fe419f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:08.574 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.587+0000 7f6fe939b700 1 -- 172.21.15.17:0/4048563501 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd40095f0 con 0x7f6fe4072ee0 2023-07-14T11:59:08.574 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.587+0000 7f6fe939b700 1 --2- 172.21.15.17:0/4048563501 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fe4072ee0 0x7f6fe4197c50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6fe000e970 tx=0x7f6fe000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:08.574 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.588+0000 7f6fda7fc700 1 -- 172.21.15.17:0/4048563501 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe0009790 con 0x7f6fe4072ee0 2023-07-14T11:59:08.574 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.588+0000 7f6feb5ff700 1 -- 172.21.15.17:0/4048563501 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fdc009c40 con 0x7f6fe4072ee0 2023-07-14T11:59:08.574 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.588+0000 7f6fda7fc700 1 -- 172.21.15.17:0/4048563501 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fe000eec0 con 0x7f6fe4072ee0 2023-07-14T11:59:08.574 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.588+0000 7f6fda7fc700 1 -- 172.21.15.17:0/4048563501 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe0003cf0 con 0x7f6fe4072ee0 2023-07-14T11:59:08.575 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.588+0000 7f6feb5ff700 1 -- 172.21.15.17:0/4048563501 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fe41a0170 con 0x7f6fe4072ee0 2023-07-14T11:59:08.575 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.589+0000 7f6feb5ff700 1 -- 172.21.15.17:0/4048563501 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fe41061b0 con 0x7f6fe4072ee0 2023-07-14T11:59:08.575 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.591+0000 7f6fda7fc700 1 -- 172.21.15.17:0/4048563501 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6fe000ccd0 con 0x7f6fe4072ee0 2023-07-14T11:59:08.576 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.592+0000 7f6fda7fc700 1 --2- 172.21.15.17:0/4048563501 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f6fd00a11d0 0x7f6fd00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:08.576 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.592+0000 7f6fda7fc700 1 -- 172.21.15.17:0/4048563501 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6fe0014070 con 0x7f6fe4072ee0 2023-07-14T11:59:08.576 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.592+0000 7f6fe8b9a700 1 --2- 172.21.15.17:0/4048563501 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f6fd00a11d0 0x7f6fd00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:08.576 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.593+0000 7f6fe8b9a700 1 --2- 172.21.15.17:0/4048563501 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f6fd00a11d0 0x7f6fd00a3680 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4005fd0 tx=0x7f6fd4009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:08.577 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.596+0000 7f6fda7fc700 1 -- 172.21.15.17:0/4048563501 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6fe0096210 con 0x7f6fe4072ee0 2023-07-14T11:59:08.834 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.853+0000 7f6feb5ff700 1 -- 172.21.15.17:0/4048563501 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6fe402c850 con 0x7f6fe4072ee0 2023-07-14T11:59:08.835 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.854+0000 7f6fda7fc700 1 -- 172.21.15.17:0/4048563501 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+501 (secure 0 0 0) 0x7f6fe0099830 con 0x7f6fe4072ee0 2023-07-14T11:59:08.835 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:59:08.835 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-07-14T11:59:08.837 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.856+0000 7f6feb5ff700 1 -- 172.21.15.17:0/4048563501 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f6fd00a11d0 msgr2=0x7f6fd00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:08.837 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.856+0000 7f6feb5ff700 1 --2- 172.21.15.17:0/4048563501 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f6fd00a11d0 0x7f6fd00a3680 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4005fd0 tx=0x7f6fd4009310 comp rx=0 tx=0).stop 2023-07-14T11:59:08.837 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.856+0000 7f6feb5ff700 1 -- 172.21.15.17:0/4048563501 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fe4072ee0 msgr2=0x7f6fe4197c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:08.837 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.856+0000 7f6feb5ff700 1 --2- 172.21.15.17:0/4048563501 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fe4072ee0 0x7f6fe4197c50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6fe000e970 tx=0x7f6fe000ec80 comp rx=0 tx=0).stop 2023-07-14T11:59:08.837 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.856+0000 7f6feb5ff700 1 -- 172.21.15.17:0/4048563501 shutdown_connections 2023-07-14T11:59:08.838 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.856+0000 7f6feb5ff700 1 --2- 172.21.15.17:0/4048563501 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f6fd00a11d0 0x7f6fd00a3680 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:08.838 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.856+0000 7f6feb5ff700 1 --2- 172.21.15.17:0/4048563501 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6fe4107a20 0x7f6fe419f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:08.838 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.856+0000 7f6feb5ff700 1 --2- 172.21.15.17:0/4048563501 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fe4072ee0 0x7f6fe4197c50 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:08.838 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.856+0000 7f6feb5ff700 1 --2- 172.21.15.17:0/4048563501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fe40744b0 0x7f6fe4198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:08.838 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.856+0000 7f6feb5ff700 1 -- 172.21.15.17:0/4048563501 >> 172.21.15.17:0/4048563501 conn(0x7f6fe40fb1c0 msgr2=0x7f6fe40fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:08.839 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.856+0000 7f6feb5ff700 1 -- 172.21.15.17:0/4048563501 shutdown_connections 2023-07-14T11:59:08.839 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:08.856+0000 7f6feb5ff700 1 -- 172.21.15.17:0/4048563501 wait complete. 2023-07-14T11:59:08.839 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 1 2023-07-14T11:59:09.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:08 smithi093 ceph-mon[128922]: Deploying daemon mds.l on smithi097 2023-07-14T11:59:09.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:08 smithi093 ceph-mon[128922]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:09.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:08 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/3985388236' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:59:09.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:08 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/479441687' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:59:09.120 INFO:journalctl@ceph.mds.l.smithi097.stdout:Jul 14 11:59:08 smithi097 systemd[1]: Starting Ceph mds.l for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T11:59:09.121 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:08 smithi097 ceph-mon[129313]: Deploying daemon mds.l on smithi097 2023-07-14T11:59:09.121 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:08 smithi097 ceph-mon[129313]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:09.121 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:08 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/3985388236' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:59:09.121 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:08 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/479441687' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:59:09.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:08 smithi017 ceph-mon[126339]: Deploying daemon mds.l on smithi097 2023-07-14T11:59:09.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:08 smithi017 ceph-mon[126339]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:09.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:08 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/3985388236' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:59:09.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:08 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/479441687' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:59:09.384 INFO:journalctl@ceph.mds.l.smithi097.stdout:Jul 14 11:59:09 smithi097 podman[154638]: 2023-07-14T11:59:09.754 INFO:journalctl@ceph.mds.l.smithi097.stdout:Jul 14 11:59:09 smithi097 ceph-mds[154678]: -- 172.21.15.97:0/3953515192 <== mon.1 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5642d91a11e0 con 0x5642d9176800 2023-07-14T11:59:09.755 INFO:journalctl@ceph.mds.l.smithi097.stdout:Jul 14 11:59:09 smithi097 bash[154638]: ec425c55ba1ce464bfef656b5c9e941e50404f331f49b4067f21f0c033fd4403 2023-07-14T11:59:09.755 INFO:journalctl@ceph.mds.l.smithi097.stdout:Jul 14 11:59:09 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-l[154654]: starting mds.l at 2023-07-14T11:59:09.755 INFO:journalctl@ceph.mds.l.smithi097.stdout:Jul 14 11:59:09 smithi097 systemd[1]: Started Ceph mds.l for b3c386cc-223c-11ee-9b33-001a4aab830c. 2023-07-14T11:59:10.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:09 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/4048563501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:59:10.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:09 smithi093 ceph-mon[128922]: mds.? [v2:172.21.15.97:6832/878913679,v1:172.21.15.97:6833/878913679] up:boot 2023-07-14T11:59:10.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:09 smithi093 ceph-mon[128922]: fsmap 1 up:standby 2023-07-14T11:59:10.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:09 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-14T11:59:10.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:09 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:10.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:09 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:10.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:09 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:10.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:09 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:59:10.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:09 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:59:10.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:09 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:59:10.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:09 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/4048563501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:59:10.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:09 smithi097 ceph-mon[129313]: mds.? [v2:172.21.15.97:6832/878913679,v1:172.21.15.97:6833/878913679] up:boot 2023-07-14T11:59:10.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:09 smithi097 ceph-mon[129313]: fsmap 1 up:standby 2023-07-14T11:59:10.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:09 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-14T11:59:10.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:09 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:10.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:09 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:10.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:09 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:10.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:09 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:59:10.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:09 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:59:10.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:09 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:59:10.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:09 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/4048563501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:59:10.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:09 smithi017 ceph-mon[126339]: mds.? [v2:172.21.15.97:6832/878913679,v1:172.21.15.97:6833/878913679] up:boot 2023-07-14T11:59:10.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:09 smithi017 ceph-mon[126339]: fsmap 1 up:standby 2023-07-14T11:59:10.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:09 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-14T11:59:10.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:09 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:10.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:09 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:10.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:09 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:10.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:09 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:59:10.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:09 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:59:10.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:09 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:59:10.797 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:10 smithi093 ceph-mon[128922]: Deploying daemon mds.k on smithi093 2023-07-14T11:59:10.798 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:10 smithi093 ceph-mon[128922]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:10.857 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:59:11.073 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.092+0000 7f6f4c1ff700 1 -- 172.21.15.17:0/1589028144 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f44100e30 msgr2=0x7f6f441012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:11.074 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.092+0000 7f6f4c1ff700 1 --2- 172.21.15.17:0/1589028144 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f44100e30 0x7f6f441012a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6f34009990 tx=0x7f6f34009ca0 comp rx=0 tx=0).stop 2023-07-14T11:59:11.074 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.092+0000 7f6f4c1ff700 1 -- 172.21.15.17:0/1589028144 shutdown_connections 2023-07-14T11:59:11.074 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.092+0000 7f6f4c1ff700 1 --2- 172.21.15.17:0/1589028144 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6f441017e0 0x7f6f44108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:11.074 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.092+0000 7f6f4c1ff700 1 --2- 172.21.15.17:0/1589028144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f440ffba0 0x7f6f440fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:11.074 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.092+0000 7f6f4c1ff700 1 --2- 172.21.15.17:0/1589028144 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f44100e30 0x7f6f441012a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:11.075 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.092+0000 7f6f4c1ff700 1 -- 172.21.15.17:0/1589028144 >> 172.21.15.17:0/1589028144 conn(0x7f6f440fb1c0 msgr2=0x7f6f440fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:11.075 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.092+0000 7f6f4c1ff700 1 -- 172.21.15.17:0/1589028144 shutdown_connections 2023-07-14T11:59:11.075 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.092+0000 7f6f4c1ff700 1 -- 172.21.15.17:0/1589028144 wait complete. 2023-07-14T11:59:11.077 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.093+0000 7f6f4c1ff700 1 Processor -- start 2023-07-14T11:59:11.077 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.093+0000 7f6f4c1ff700 1 -- start start 2023-07-14T11:59:11.077 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.094+0000 7f6f4c1ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f440ffba0 0x7f6f440742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:11.077 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.094+0000 7f6f4c1ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f44100e30 0x7f6f44072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:11.077 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.094+0000 7f6f4c1ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6f441017e0 0x7f6f44072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:11.077 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.094+0000 7f6f4c1ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f440735e0 con 0x7f6f440ffba0 2023-07-14T11:59:11.078 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.094+0000 7f6f4c1ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f44073750 con 0x7f6f441017e0 2023-07-14T11:59:11.078 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.094+0000 7f6f4c1ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4419f900 con 0x7f6f44100e30 2023-07-14T11:59:11.078 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.094+0000 7f6f49f9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f440ffba0 0x7f6f440742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:11.078 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.094+0000 7f6f4979a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f44100e30 0x7f6f44072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:11.078 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.094+0000 7f6f49f9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f440ffba0 0x7f6f440742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36780/0 (socket says 172.21.15.17:36780) 2023-07-14T11:59:11.078 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.094+0000 7f6f49f9b700 1 -- 172.21.15.17:0/260692171 learned_addr learned my addr 172.21.15.17:0/260692171 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:59:11.078 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.094+0000 7f6f4a79c700 1 --2- 172.21.15.17:0/260692171 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6f441017e0 0x7f6f44072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:11.078 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.094+0000 7f6f49f9b700 1 -- 172.21.15.17:0/260692171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f44100e30 msgr2=0x7f6f44072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:11.079 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.094+0000 7f6f49f9b700 1 --2- 172.21.15.17:0/260692171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f44100e30 0x7f6f44072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:11.079 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.094+0000 7f6f49f9b700 1 -- 172.21.15.17:0/260692171 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6f441017e0 msgr2=0x7f6f44072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:11.079 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.094+0000 7f6f49f9b700 1 --2- 172.21.15.17:0/260692171 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6f441017e0 0x7f6f44072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:11.079 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.094+0000 7f6f49f9b700 1 -- 172.21.15.17:0/260692171 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f40009520 con 0x7f6f440ffba0 2023-07-14T11:59:11.079 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.094+0000 7f6f49f9b700 1 --2- 172.21.15.17:0/260692171 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f440ffba0 0x7f6f440742c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6f4000eb30 tx=0x7f6f4000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:11.079 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.095+0000 7f6f3affd700 1 -- 172.21.15.17:0/260692171 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f40009880 con 0x7f6f440ffba0 2023-07-14T11:59:11.079 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.095+0000 7f6f3affd700 1 -- 172.21.15.17:0/260692171 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f400044e0 con 0x7f6f440ffba0 2023-07-14T11:59:11.080 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.095+0000 7f6f3affd700 1 -- 172.21.15.17:0/260692171 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f40003d80 con 0x7f6f440ffba0 2023-07-14T11:59:11.080 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.095+0000 7f6f4c1ff700 1 -- 172.21.15.17:0/260692171 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f340095f0 con 0x7f6f440ffba0 2023-07-14T11:59:11.080 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.095+0000 7f6f4c1ff700 1 -- 172.21.15.17:0/260692171 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f4419ff10 con 0x7f6f440ffba0 2023-07-14T11:59:11.080 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.096+0000 7f6f4c1ff700 1 -- 172.21.15.17:0/260692171 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f44106bc0 con 0x7f6f440ffba0 2023-07-14T11:59:11.081 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.098+0000 7f6f3affd700 1 -- 172.21.15.17:0/260692171 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6f40003f20 con 0x7f6f440ffba0 2023-07-14T11:59:11.081 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.099+0000 7f6f3affd700 1 --2- 172.21.15.17:0/260692171 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f6f300a11d0 0x7f6f300a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:11.081 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.100+0000 7f6f3affd700 1 -- 172.21.15.17:0/260692171 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6f40014070 con 0x7f6f440ffba0 2023-07-14T11:59:11.081 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.100+0000 7f6f4979a700 1 --2- 172.21.15.17:0/260692171 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f6f300a11d0 0x7f6f300a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:11.082 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.100+0000 7f6f4979a700 1 --2- 172.21.15.17:0/260692171 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f6f300a11d0 0x7f6f300a3680 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6f3400b680 tx=0x7f6f34005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:11.084 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.103+0000 7f6f3affd700 1 -- 172.21.15.17:0/260692171 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6f40096530 con 0x7f6f440ffba0 2023-07-14T11:59:11.111 INFO:journalctl@ceph.mds.k.smithi093.stdout:Jul 14 11:59:11 smithi093 systemd[1]: Starting Ceph mds.k for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T11:59:11.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:10 smithi097 ceph-mon[129313]: Deploying daemon mds.k on smithi093 2023-07-14T11:59:11.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:10 smithi097 ceph-mon[129313]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:11.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:10 smithi017 ceph-mon[126339]: Deploying daemon mds.k on smithi093 2023-07-14T11:59:11.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:10 smithi017 ceph-mon[126339]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:11.341 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.360+0000 7f6f4c1ff700 1 -- 172.21.15.17:0/260692171 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6f4419fac0 con 0x7f6f440ffba0 2023-07-14T11:59:11.341 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.360+0000 7f6f3affd700 1 -- 172.21.15.17:0/260692171 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1254 (secure 0 0 0) 0x7f6f40099b50 con 0x7f6f440ffba0 2023-07-14T11:59:11.342 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:59:11.342 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/878913679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":878913679},{"type":"v1","addr":"172.21.15.97:6833","nonce":878913679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-07-14T11:59:11.344 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.363+0000 7f6f4c1ff700 1 -- 172.21.15.17:0/260692171 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f6f300a11d0 msgr2=0x7f6f300a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:11.344 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.363+0000 7f6f4c1ff700 1 --2- 172.21.15.17:0/260692171 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f6f300a11d0 0x7f6f300a3680 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6f3400b680 tx=0x7f6f34005fd0 comp rx=0 tx=0).stop 2023-07-14T11:59:11.344 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.363+0000 7f6f4c1ff700 1 -- 172.21.15.17:0/260692171 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f440ffba0 msgr2=0x7f6f440742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:11.344 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.363+0000 7f6f4c1ff700 1 --2- 172.21.15.17:0/260692171 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f440ffba0 0x7f6f440742c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6f4000eb30 tx=0x7f6f4000ee40 comp rx=0 tx=0).stop 2023-07-14T11:59:11.345 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.363+0000 7f6f4c1ff700 1 -- 172.21.15.17:0/260692171 shutdown_connections 2023-07-14T11:59:11.345 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.363+0000 7f6f4c1ff700 1 --2- 172.21.15.17:0/260692171 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f6f300a11d0 0x7f6f300a3680 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:11.345 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.363+0000 7f6f4c1ff700 1 --2- 172.21.15.17:0/260692171 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6f441017e0 0x7f6f44072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:11.345 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.363+0000 7f6f4c1ff700 1 --2- 172.21.15.17:0/260692171 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f440ffba0 0x7f6f440742c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:11.345 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.363+0000 7f6f4c1ff700 1 --2- 172.21.15.17:0/260692171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f44100e30 0x7f6f44072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:11.345 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.363+0000 7f6f4c1ff700 1 -- 172.21.15.17:0/260692171 >> 172.21.15.17:0/260692171 conn(0x7f6f440fb1c0 msgr2=0x7f6f440fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:11.345 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.363+0000 7f6f4c1ff700 1 -- 172.21.15.17:0/260692171 shutdown_connections 2023-07-14T11:59:11.346 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:11.363+0000 7f6f4c1ff700 1 -- 172.21.15.17:0/260692171 wait complete. 2023-07-14T11:59:11.346 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 2 2023-07-14T11:59:11.603 INFO:journalctl@ceph.mds.k.smithi093.stdout:Jul 14 11:59:11 smithi093 podman[153434]: 2023-07-14T11:59:11.861 INFO:journalctl@ceph.mds.k.smithi093.stdout:Jul 14 11:59:11 smithi093 ceph-mds[153474]: -- 172.21.15.93:0/3615458489 <== mon.2 v2:172.21.15.93:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5561ab0351e0 con 0x5561ab00a800 2023-07-14T11:59:11.861 INFO:journalctl@ceph.mds.k.smithi093.stdout:Jul 14 11:59:11 smithi093 bash[153434]: 6e44de260e66c0128cb5d33da4f93147c9084231680354f923a6121ca40eff13 2023-07-14T11:59:11.862 INFO:journalctl@ceph.mds.k.smithi093.stdout:Jul 14 11:59:11 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-k[153450]: starting mds.k at 2023-07-14T11:59:11.862 INFO:journalctl@ceph.mds.k.smithi093.stdout:Jul 14 11:59:11 smithi093 systemd[1]: Started Ceph mds.k for b3c386cc-223c-11ee-9b33-001a4aab830c. 2023-07-14T11:59:11.862 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:11 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/260692171' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:59:11.862 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:11 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:11.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:11 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:11.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:11 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:11.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:11 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:59:11.863 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:11 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:59:11.864 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:11 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:59:12.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:11 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/260692171' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:59:12.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:11 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:12.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:11 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:12.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:11 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:12.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:11 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:59:12.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:11 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:59:12.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:11 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:59:12.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:11 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/260692171' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:59:12.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:11 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:12.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:11 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:12.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:11 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:12.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:11 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:59:12.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:11 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:59:12.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:11 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:59:12.964 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:12 smithi017 ceph-mon[126339]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:12.965 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:12 smithi017 ceph-mon[126339]: Deploying daemon mds.j on smithi017 2023-07-14T11:59:12.965 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:12 smithi017 ceph-mon[126339]: mds.? [v2:172.21.15.93:6832/326319712,v1:172.21.15.93:6833/326319712] up:boot 2023-07-14T11:59:12.965 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:12 smithi017 ceph-mon[126339]: fsmap 2 up:standby 2023-07-14T11:59:12.965 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:12 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-14T11:59:13.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:12 smithi093 ceph-mon[128922]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:13.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:12 smithi093 ceph-mon[128922]: Deploying daemon mds.j on smithi017 2023-07-14T11:59:13.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:12 smithi093 ceph-mon[128922]: mds.? [v2:172.21.15.93:6832/326319712,v1:172.21.15.93:6833/326319712] up:boot 2023-07-14T11:59:13.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:12 smithi093 ceph-mon[128922]: fsmap 2 up:standby 2023-07-14T11:59:13.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:12 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-14T11:59:13.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:12 smithi097 ceph-mon[129313]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:13.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:12 smithi097 ceph-mon[129313]: Deploying daemon mds.j on smithi017 2023-07-14T11:59:13.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:12 smithi097 ceph-mon[129313]: mds.? [v2:172.21.15.93:6832/326319712,v1:172.21.15.93:6833/326319712] up:boot 2023-07-14T11:59:13.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:12 smithi097 ceph-mon[129313]: fsmap 2 up:standby 2023-07-14T11:59:13.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:12 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-14T11:59:13.363 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:59:13.592 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.611+0000 7fbe5d5ff700 1 -- 172.21.15.17:0/220253015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe58072ee0 msgr2=0x7fbe58073350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:13.593 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.611+0000 7fbe5d5ff700 1 --2- 172.21.15.17:0/220253015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe58072ee0 0x7fbe58073350 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c009990 tx=0x7fbe4c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:59:13.593 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.611+0000 7fbe5d5ff700 1 -- 172.21.15.17:0/220253015 shutdown_connections 2023-07-14T11:59:13.593 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.611+0000 7fbe5d5ff700 1 --2- 172.21.15.17:0/220253015 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe58107a20 0x7fbe58109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:13.593 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.611+0000 7fbe5d5ff700 1 --2- 172.21.15.17:0/220253015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe580744b0 0x7fbe580729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:13.593 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.611+0000 7fbe5d5ff700 1 --2- 172.21.15.17:0/220253015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe58072ee0 0x7fbe58073350 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:13.593 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.611+0000 7fbe5d5ff700 1 -- 172.21.15.17:0/220253015 >> 172.21.15.17:0/220253015 conn(0x7fbe580fb1c0 msgr2=0x7fbe580fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:13.594 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.612+0000 7fbe5d5ff700 1 -- 172.21.15.17:0/220253015 shutdown_connections 2023-07-14T11:59:13.594 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.612+0000 7fbe5d5ff700 1 -- 172.21.15.17:0/220253015 wait complete. 2023-07-14T11:59:13.595 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.613+0000 7fbe5d5ff700 1 Processor -- start 2023-07-14T11:59:13.595 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.613+0000 7fbe5d5ff700 1 -- start start 2023-07-14T11:59:13.595 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.613+0000 7fbe5d5ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe58072ee0 0x7fbe58197cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:13.596 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.613+0000 7fbe5d5ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe580744b0 0x7fbe581981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:13.596 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.613+0000 7fbe5d5ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe58107a20 0x7fbe5819f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:13.596 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.613+0000 7fbe5d5ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5819f820 con 0x7fbe58107a20 2023-07-14T11:59:13.596 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.613+0000 7fbe5d5ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5819f990 con 0x7fbe580744b0 2023-07-14T11:59:13.596 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.613+0000 7fbe5d5ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5819fb50 con 0x7fbe58072ee0 2023-07-14T11:59:13.596 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.614+0000 7fbe567fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe580744b0 0x7fbe581981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:13.596 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.614+0000 7fbe567fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe580744b0 0x7fbe581981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.17:33948/0 (socket says 172.21.15.17:33948) 2023-07-14T11:59:13.597 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.614+0000 7fbe567fc700 1 -- 172.21.15.17:0/3929617103 learned_addr learned my addr 172.21.15.17:0/3929617103 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:59:13.597 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.614+0000 7fbe577fe700 1 --2- 172.21.15.17:0/3929617103 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe58107a20 0x7fbe5819f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:13.597 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.614+0000 7fbe56ffd700 1 --2- 172.21.15.17:0/3929617103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe58072ee0 0x7fbe58197cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:13.597 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.614+0000 7fbe567fc700 1 -- 172.21.15.17:0/3929617103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe58072ee0 msgr2=0x7fbe58197cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:13.597 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.614+0000 7fbe567fc700 1 --2- 172.21.15.17:0/3929617103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe58072ee0 0x7fbe58197cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:13.597 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.614+0000 7fbe567fc700 1 -- 172.21.15.17:0/3929617103 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe58107a20 msgr2=0x7fbe5819f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:13.597 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.614+0000 7fbe567fc700 1 --2- 172.21.15.17:0/3929617103 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe58107a20 0x7fbe5819f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:13.597 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.614+0000 7fbe567fc700 1 -- 172.21.15.17:0/3929617103 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe4c0095f0 con 0x7fbe580744b0 2023-07-14T11:59:13.598 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.615+0000 7fbe567fc700 1 --2- 172.21.15.17:0/3929617103 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe580744b0 0x7fbe581981f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c00b680 tx=0x7fbe4c005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:13.598 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.615+0000 7fbe3ffff700 1 -- 172.21.15.17:0/3929617103 <== mon.2 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c01d070 con 0x7fbe580744b0 2023-07-14T11:59:13.599 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.615+0000 7fbe3ffff700 1 -- 172.21.15.17:0/3929617103 <== mon.2 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe4c022470 con 0x7fbe580744b0 2023-07-14T11:59:13.599 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.615+0000 7fbe3ffff700 1 -- 172.21.15.17:0/3929617103 <== mon.2 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c017650 con 0x7fbe580744b0 2023-07-14T11:59:13.599 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.615+0000 7fbe5d5ff700 1 -- 172.21.15.17:0/3929617103 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe5819fdd0 con 0x7fbe580744b0 2023-07-14T11:59:13.599 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.615+0000 7fbe5d5ff700 1 -- 172.21.15.17:0/3929617103 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe581a01a0 con 0x7fbe580744b0 2023-07-14T11:59:13.599 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.616+0000 7fbe5d5ff700 1 -- 172.21.15.17:0/3929617103 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe5810d5f0 con 0x7fbe580744b0 2023-07-14T11:59:13.600 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.619+0000 7fbe3ffff700 1 -- 172.21.15.17:0/3929617103 <== mon.2 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fbe4c0225e0 con 0x7fbe580744b0 2023-07-14T11:59:13.601 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.621+0000 7fbe3ffff700 1 --2- 172.21.15.17:0/3929617103 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fbe400a11d0 0x7fbe400a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:13.602 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.621+0000 7fbe3ffff700 1 -- 172.21.15.17:0/3929617103 <== mon.2 v2:172.21.15.93:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbe4c0cda70 con 0x7fbe580744b0 2023-07-14T11:59:13.602 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.621+0000 7fbe56ffd700 1 --2- 172.21.15.17:0/3929617103 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fbe400a11d0 0x7fbe400a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:13.603 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.622+0000 7fbe56ffd700 1 --2- 172.21.15.17:0/3929617103 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fbe400a11d0 0x7fbe400a3680 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbe58198b80 tx=0x7fbe48009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:13.603 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.622+0000 7fbe3ffff700 1 -- 172.21.15.17:0/3929617103 <== mon.2 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fbe4c0976d0 con 0x7fbe580744b0 2023-07-14T11:59:13.883 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.902+0000 7fbe5d5ff700 1 -- 172.21.15.17:0/3929617103 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe581a07b0 con 0x7fbe580744b0 2023-07-14T11:59:13.884 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.903+0000 7fbe3ffff700 1 -- 172.21.15.17:0/3929617103 <== mon.2 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2008 (secure 0 0 0) 0x7fbe4c0277f0 con 0x7fbe580744b0 2023-07-14T11:59:13.884 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:59:13.884 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/326319712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":326319712},{"type":"v1","addr":"172.21.15.93:6833","nonce":326319712}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/878913679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":878913679},{"type":"v1","addr":"172.21.15.97:6833","nonce":878913679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-07-14T11:59:13.888 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.906+0000 7fbe5d5ff700 1 -- 172.21.15.17:0/3929617103 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fbe400a11d0 msgr2=0x7fbe400a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:13.888 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.906+0000 7fbe5d5ff700 1 --2- 172.21.15.17:0/3929617103 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fbe400a11d0 0x7fbe400a3680 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbe58198b80 tx=0x7fbe48009260 comp rx=0 tx=0).stop 2023-07-14T11:59:13.888 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.907+0000 7fbe5d5ff700 1 -- 172.21.15.17:0/3929617103 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe580744b0 msgr2=0x7fbe581981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:13.888 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.907+0000 7fbe5d5ff700 1 --2- 172.21.15.17:0/3929617103 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe580744b0 0x7fbe581981f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c00b680 tx=0x7fbe4c005740 comp rx=0 tx=0).stop 2023-07-14T11:59:13.889 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.907+0000 7fbe5d5ff700 1 -- 172.21.15.17:0/3929617103 shutdown_connections 2023-07-14T11:59:13.889 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.907+0000 7fbe5d5ff700 1 --2- 172.21.15.17:0/3929617103 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fbe400a11d0 0x7fbe400a3680 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:13.889 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.907+0000 7fbe5d5ff700 1 --2- 172.21.15.17:0/3929617103 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe58107a20 0x7fbe5819f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:13.889 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.907+0000 7fbe5d5ff700 1 --2- 172.21.15.17:0/3929617103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe58072ee0 0x7fbe58197cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:13.889 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.907+0000 7fbe5d5ff700 1 --2- 172.21.15.17:0/3929617103 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe580744b0 0x7fbe581981f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:13.889 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.907+0000 7fbe5d5ff700 1 -- 172.21.15.17:0/3929617103 >> 172.21.15.17:0/3929617103 conn(0x7fbe580fb1c0 msgr2=0x7fbe580fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:13.889 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.907+0000 7fbe5d5ff700 1 -- 172.21.15.17:0/3929617103 shutdown_connections 2023-07-14T11:59:13.890 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:13.907+0000 7fbe5d5ff700 1 -- 172.21.15.17:0/3929617103 wait complete. 2023-07-14T11:59:13.890 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 3 2023-07-14T11:59:14.395 INFO:journalctl@ceph.mds.j.smithi017.stdout:Jul 14 11:59:14 smithi017 systemd[1]: Starting Ceph mds.j for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T11:59:14.715 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:14 smithi017 ceph-mon[126339]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:14.715 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:14 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/3929617103' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:59:14.716 INFO:journalctl@ceph.mds.j.smithi017.stdout:Jul 14 11:59:14 smithi017 podman[159772]: 2023-07-14T11:59:14.861 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:14 smithi093 ceph-mon[128922]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:14.861 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:14 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/3929617103' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:59:14.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:14 smithi097 ceph-mon[129313]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:14.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:14 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/3929617103' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:59:15.144 INFO:journalctl@ceph.mds.j.smithi017.stdout:Jul 14 11:59:14 smithi017 bash[159772]: be4008277c6aeee38cd37295c38d49fc8f3d3046f1194356e6958d8a558da1f5 2023-07-14T11:59:15.144 INFO:journalctl@ceph.mds.j.smithi017.stdout:Jul 14 11:59:14 smithi017 ceph-mds[159812]: -- 172.21.15.17:0/633834765 <== mon.0 v2:172.21.15.17:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561b437091e0 con 0x561b436df000 2023-07-14T11:59:15.145 INFO:journalctl@ceph.mds.j.smithi017.stdout:Jul 14 11:59:14 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-j[159788]: starting mds.j at 2023-07-14T11:59:15.145 INFO:journalctl@ceph.mds.j.smithi017.stdout:Jul 14 11:59:14 smithi017 systemd[1]: Started Ceph mds.j for b3c386cc-223c-11ee-9b33-001a4aab830c. 2023-07-14T11:59:15.917 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:59:16.134 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.152+0000 7f684cbff700 1 -- 172.21.15.17:0/2840670908 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6848100e30 msgr2=0x7f68481012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:16.134 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.152+0000 7f684cbff700 1 --2- 172.21.15.17:0/2840670908 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6848100e30 0x7f68481012a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f683c009990 tx=0x7f683c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:59:16.135 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.152+0000 7f684cbff700 1 -- 172.21.15.17:0/2840670908 shutdown_connections 2023-07-14T11:59:16.135 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.152+0000 7f684cbff700 1 --2- 172.21.15.17:0/2840670908 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68481017e0 0x7f6848108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:16.135 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.153+0000 7f684cbff700 1 --2- 172.21.15.17:0/2840670908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68480ffba0 0x7f68480fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:16.136 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.153+0000 7f684cbff700 1 --2- 172.21.15.17:0/2840670908 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6848100e30 0x7f68481012a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:16.136 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.153+0000 7f684cbff700 1 -- 172.21.15.17:0/2840670908 >> 172.21.15.17:0/2840670908 conn(0x7f68480fb1c0 msgr2=0x7f68480fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:16.136 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.153+0000 7f684cbff700 1 -- 172.21.15.17:0/2840670908 shutdown_connections 2023-07-14T11:59:16.136 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.153+0000 7f684cbff700 1 -- 172.21.15.17:0/2840670908 wait complete. 2023-07-14T11:59:16.138 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.154+0000 7f684cbff700 1 Processor -- start 2023-07-14T11:59:16.138 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.154+0000 7f684cbff700 1 -- start start 2023-07-14T11:59:16.138 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.154+0000 7f684cbff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68480ffba0 0x7f6848197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:16.138 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.154+0000 7f684cbff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6848100e30 0x7f6848198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:16.139 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.154+0000 7f684cbff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f68481017e0 0x7f684819f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:16.139 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.154+0000 7f684cbff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684819f660 con 0x7f6848100e30 2023-07-14T11:59:16.139 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.154+0000 7f684cbff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684819f7a0 con 0x7f68481017e0 2023-07-14T11:59:16.139 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.154+0000 7f684cbff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684819f930 con 0x7f68480ffba0 2023-07-14T11:59:16.139 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.154+0000 7f6845d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6848100e30 0x7f6848198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:16.139 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.154+0000 7f6845d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6848100e30 0x7f6848198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34990/0 (socket says 172.21.15.17:34990) 2023-07-14T11:59:16.140 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.154+0000 7f6845d9b700 1 -- 172.21.15.17:0/811188435 learned_addr learned my addr 172.21.15.17:0/811188435 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:59:16.140 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.154+0000 7f684659c700 1 --2- 172.21.15.17:0/811188435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68480ffba0 0x7f6848197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:16.140 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.155+0000 7f6846d9d700 1 --2- 172.21.15.17:0/811188435 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f68481017e0 0x7f684819f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:16.140 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.155+0000 7f6845d9b700 1 -- 172.21.15.17:0/811188435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68480ffba0 msgr2=0x7f6848197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:16.140 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.155+0000 7f6845d9b700 1 --2- 172.21.15.17:0/811188435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68480ffba0 0x7f6848197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:16.141 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.155+0000 7f6845d9b700 1 -- 172.21.15.17:0/811188435 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f68481017e0 msgr2=0x7f684819f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:16.141 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.155+0000 7f6845d9b700 1 --2- 172.21.15.17:0/811188435 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f68481017e0 0x7f684819f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:16.141 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.155+0000 7f6845d9b700 1 -- 172.21.15.17:0/811188435 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f683c0095f0 con 0x7f6848100e30 2023-07-14T11:59:16.141 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.155+0000 7f6845d9b700 1 --2- 172.21.15.17:0/811188435 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6848100e30 0x7f6848198020 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f683c004740 tx=0x7f683c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:16.141 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.155+0000 7f68377fe700 1 -- 172.21.15.17:0/811188435 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683c01d070 con 0x7f6848100e30 2023-07-14T11:59:16.141 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.155+0000 7f684cbff700 1 -- 172.21.15.17:0/811188435 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f684819fbb0 con 0x7f6848100e30 2023-07-14T11:59:16.141 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.155+0000 7f68377fe700 1 -- 172.21.15.17:0/811188435 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f683c004830 con 0x7f6848100e30 2023-07-14T11:59:16.141 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.155+0000 7f68377fe700 1 -- 172.21.15.17:0/811188435 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683c0053c0 con 0x7f6848100e30 2023-07-14T11:59:16.142 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.156+0000 7f684cbff700 1 -- 172.21.15.17:0/811188435 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f684819ffd0 con 0x7f6848100e30 2023-07-14T11:59:16.142 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.157+0000 7f684cbff700 1 -- 172.21.15.17:0/811188435 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f684810b3c0 con 0x7f6848100e30 2023-07-14T11:59:16.142 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.159+0000 7f68377fe700 1 -- 172.21.15.17:0/811188435 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f683c005580 con 0x7f6848100e30 2023-07-14T11:59:16.143 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.160+0000 7f68377fe700 1 --2- 172.21.15.17:0/811188435 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f68300a1150 0x7f68300a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:16.143 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.160+0000 7f68377fe700 1 -- 172.21.15.17:0/811188435 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f683c0cd650 con 0x7f6848100e30 2023-07-14T11:59:16.143 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.160+0000 7f684659c700 1 --2- 172.21.15.17:0/811188435 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f68300a1150 0x7f68300a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:16.143 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.161+0000 7f684659c700 1 --2- 172.21.15.17:0/811188435 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f68300a1150 0x7f68300a3600 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6838009c60 tx=0x7f6838009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:16.149 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.169+0000 7f68377fe700 1 -- 172.21.15.17:0/811188435 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f683c05a790 con 0x7f6848100e30 2023-07-14T11:59:16.247 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:15 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:16.248 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:15 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:16.248 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:15 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:16.248 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:15 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:59:16.248 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:15 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:59:16.248 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:15 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:59:16.249 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:15 smithi017 ceph-mon[126339]: Deploying daemon mds.d on smithi017 2023-07-14T11:59:16.249 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:15 smithi017 ceph-mon[126339]: mds.? [v2:172.21.15.17:6834/55384193,v1:172.21.15.17:6835/55384193] up:boot 2023-07-14T11:59:16.249 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:15 smithi017 ceph-mon[126339]: fsmap 3 up:standby 2023-07-14T11:59:16.249 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:15 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-14T11:59:16.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:15 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:16.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:15 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:16.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:15 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:16.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:15 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:59:16.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:15 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:59:16.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:15 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:59:16.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:15 smithi093 ceph-mon[128922]: Deploying daemon mds.d on smithi017 2023-07-14T11:59:16.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:15 smithi093 ceph-mon[128922]: mds.? [v2:172.21.15.17:6834/55384193,v1:172.21.15.17:6835/55384193] up:boot 2023-07-14T11:59:16.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:15 smithi093 ceph-mon[128922]: fsmap 3 up:standby 2023-07-14T11:59:16.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:15 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-14T11:59:16.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:15 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:16.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:15 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:16.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:15 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:16.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:15 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:59:16.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:15 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:59:16.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:15 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:59:16.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:15 smithi097 ceph-mon[129313]: Deploying daemon mds.d on smithi017 2023-07-14T11:59:16.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:15 smithi097 ceph-mon[129313]: mds.? [v2:172.21.15.17:6834/55384193,v1:172.21.15.17:6835/55384193] up:boot 2023-07-14T11:59:16.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:15 smithi097 ceph-mon[129313]: fsmap 3 up:standby 2023-07-14T11:59:16.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:15 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-14T11:59:16.414 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.433+0000 7f684cbff700 1 -- 172.21.15.17:0/811188435 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f68481a0640 con 0x7f6848100e30 2023-07-14T11:59:16.414 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.434+0000 7f68377fe700 1 -- 172.21.15.17:0/811188435 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2759 (secure 0 0 0) 0x7f683c097230 con 0x7f6848100e30 2023-07-14T11:59:16.415 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:59:16.415 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/55384193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":55384193},{"type":"v1","addr":"172.21.15.17:6835","nonce":55384193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/326319712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":326319712},{"type":"v1","addr":"172.21.15.93:6833","nonce":326319712}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/878913679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":878913679},{"type":"v1","addr":"172.21.15.97:6833","nonce":878913679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-07-14T11:59:16.417 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.436+0000 7f684cbff700 1 -- 172.21.15.17:0/811188435 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f68300a1150 msgr2=0x7f68300a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:16.417 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.436+0000 7f684cbff700 1 --2- 172.21.15.17:0/811188435 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f68300a1150 0x7f68300a3600 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6838009c60 tx=0x7f6838009260 comp rx=0 tx=0).stop 2023-07-14T11:59:16.417 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.436+0000 7f684cbff700 1 -- 172.21.15.17:0/811188435 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6848100e30 msgr2=0x7f6848198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:16.417 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.436+0000 7f684cbff700 1 --2- 172.21.15.17:0/811188435 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6848100e30 0x7f6848198020 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f683c004740 tx=0x7f683c009580 comp rx=0 tx=0).stop 2023-07-14T11:59:16.418 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.436+0000 7f684cbff700 1 -- 172.21.15.17:0/811188435 shutdown_connections 2023-07-14T11:59:16.418 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.436+0000 7f684cbff700 1 --2- 172.21.15.17:0/811188435 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f68300a1150 0x7f68300a3600 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:16.418 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.436+0000 7f684cbff700 1 --2- 172.21.15.17:0/811188435 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f68481017e0 0x7f684819f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:16.418 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.436+0000 7f684cbff700 1 --2- 172.21.15.17:0/811188435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68480ffba0 0x7f6848197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:16.418 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.436+0000 7f684cbff700 1 --2- 172.21.15.17:0/811188435 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6848100e30 0x7f6848198020 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:16.418 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.436+0000 7f684cbff700 1 -- 172.21.15.17:0/811188435 >> 172.21.15.17:0/811188435 conn(0x7f68480fb1c0 msgr2=0x7f68480fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:16.418 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.436+0000 7f684cbff700 1 -- 172.21.15.17:0/811188435 shutdown_connections 2023-07-14T11:59:16.419 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:16.436+0000 7f684cbff700 1 -- 172.21.15.17:0/811188435 wait complete. 2023-07-14T11:59:16.419 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 4 2023-07-14T11:59:17.190 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:16 smithi017 ceph-mon[126339]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:17.190 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:16 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/811188435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:59:17.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:16 smithi093 ceph-mon[128922]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:17.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:16 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/811188435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:59:17.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:16 smithi097 ceph-mon[129313]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:17.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:16 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/811188435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:59:17.644 INFO:journalctl@ceph.mds.d.smithi017.stdout:Jul 14 11:59:17 smithi017 systemd[1]: Starting Ceph mds.d for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T11:59:17.930 INFO:journalctl@ceph.mds.d.smithi017.stdout:Jul 14 11:59:17 smithi017 podman[160164]: 2023-07-14T11:59:17.931 INFO:journalctl@ceph.mds.d.smithi017.stdout:Jul 14 11:59:17 smithi017 bash[160164]: 2f4e1cd1dc5974e1a6c074e6cac3ff7b26edcdb4debd38a5ce2fd05d35206417 2023-07-14T11:59:18.394 INFO:journalctl@ceph.mds.d.smithi017.stdout:Jul 14 11:59:17 smithi017 ceph-mds[160204]: -- 172.21.15.17:0/656808980 <== mon.0 v2:172.21.15.17:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560e629191e0 con 0x560e628ee800 2023-07-14T11:59:18.394 INFO:journalctl@ceph.mds.d.smithi017.stdout:Jul 14 11:59:17 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-d[160180]: starting mds.d at 2023-07-14T11:59:18.394 INFO:journalctl@ceph.mds.d.smithi017.stdout:Jul 14 11:59:17 smithi017 systemd[1]: Started Ceph mds.d for b3c386cc-223c-11ee-9b33-001a4aab830c. 2023-07-14T11:59:18.439 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:59:18.652 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.670+0000 7ff5389ff700 1 -- 172.21.15.17:0/1695035799 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5340ffba0 msgr2=0x7ff5340fffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:18.652 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.670+0000 7ff5389ff700 1 --2- 172.21.15.17:0/1695035799 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5340ffba0 0x7ff5340fffb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff524009990 tx=0x7ff524009ca0 comp rx=0 tx=0).stop 2023-07-14T11:59:18.652 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.670+0000 7ff5389ff700 1 -- 172.21.15.17:0/1695035799 shutdown_connections 2023-07-14T11:59:18.652 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.670+0000 7ff5389ff700 1 --2- 172.21.15.17:0/1695035799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5341017e0 0x7ff534108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:18.653 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.670+0000 7ff5389ff700 1 --2- 172.21.15.17:0/1695035799 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5340ffba0 0x7ff5340fffb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:18.653 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.670+0000 7ff5389ff700 1 --2- 172.21.15.17:0/1695035799 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff534100e30 0x7ff5341012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:18.653 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.670+0000 7ff5389ff700 1 -- 172.21.15.17:0/1695035799 >> 172.21.15.17:0/1695035799 conn(0x7ff5340fb1c0 msgr2=0x7ff5340fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:18.653 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.670+0000 7ff5389ff700 1 -- 172.21.15.17:0/1695035799 shutdown_connections 2023-07-14T11:59:18.653 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.670+0000 7ff5389ff700 1 -- 172.21.15.17:0/1695035799 wait complete. 2023-07-14T11:59:18.655 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.671+0000 7ff5389ff700 1 Processor -- start 2023-07-14T11:59:18.655 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.672+0000 7ff5389ff700 1 -- start start 2023-07-14T11:59:18.655 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.672+0000 7ff5389ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff5340ffba0 0x7ff534197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:18.656 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.672+0000 7ff5389ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff534100e30 0x7ff5341980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:18.656 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.672+0000 7ff5389ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5341017e0 0x7ff53419f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:18.656 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.672+0000 7ff5389ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53419f6f0 con 0x7ff5341017e0 2023-07-14T11:59:18.656 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.672+0000 7ff5389ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53419f830 con 0x7ff5340ffba0 2023-07-14T11:59:18.656 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.672+0000 7ff5389ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53419f9c0 con 0x7ff534100e30 2023-07-14T11:59:18.656 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.672+0000 7ff53259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff5340ffba0 0x7ff534197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:18.656 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.672+0000 7ff53259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff5340ffba0 0x7ff534197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.17:34040/0 (socket says 172.21.15.17:34040) 2023-07-14T11:59:18.657 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.672+0000 7ff53259c700 1 -- 172.21.15.17:0/3701944072 learned_addr learned my addr 172.21.15.17:0/3701944072 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:59:18.657 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.672+0000 7ff532d9d700 1 --2- 172.21.15.17:0/3701944072 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5341017e0 0x7ff53419f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:18.657 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.672+0000 7ff531d9b700 1 --2- 172.21.15.17:0/3701944072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff534100e30 0x7ff5341980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:18.657 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.673+0000 7ff53259c700 1 -- 172.21.15.17:0/3701944072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff534100e30 msgr2=0x7ff5341980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:18.657 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.673+0000 7ff53259c700 1 --2- 172.21.15.17:0/3701944072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff534100e30 0x7ff5341980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:18.657 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.673+0000 7ff53259c700 1 -- 172.21.15.17:0/3701944072 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5341017e0 msgr2=0x7ff53419f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:18.658 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.673+0000 7ff53259c700 1 --2- 172.21.15.17:0/3701944072 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5341017e0 0x7ff53419f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:18.658 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.673+0000 7ff53259c700 1 -- 172.21.15.17:0/3701944072 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5240095f0 con 0x7ff5340ffba0 2023-07-14T11:59:18.659 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.673+0000 7ff531d9b700 1 --2- 172.21.15.17:0/3701944072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff534100e30 0x7ff5341980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:59:18.659 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.673+0000 7ff53259c700 1 --2- 172.21.15.17:0/3701944072 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff5340ffba0 0x7ff534197b70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff524004730 tx=0x7ff524004760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:18.659 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.673+0000 7ff5237fe700 1 -- 172.21.15.17:0/3701944072 <== mon.2 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52401d070 con 0x7ff5340ffba0 2023-07-14T11:59:18.660 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.673+0000 7ff5389ff700 1 -- 172.21.15.17:0/3701944072 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff53419fc40 con 0x7ff5340ffba0 2023-07-14T11:59:18.660 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.673+0000 7ff5237fe700 1 -- 172.21.15.17:0/3701944072 <== mon.2 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff524004c90 con 0x7ff5340ffba0 2023-07-14T11:59:18.660 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.673+0000 7ff5237fe700 1 -- 172.21.15.17:0/3701944072 <== mon.2 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff524005570 con 0x7ff5340ffba0 2023-07-14T11:59:18.661 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.673+0000 7ff5389ff700 1 -- 172.21.15.17:0/3701944072 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5341a0060 con 0x7ff5340ffba0 2023-07-14T11:59:18.661 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.674+0000 7ff5389ff700 1 -- 172.21.15.17:0/3701944072 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff53410b3c0 con 0x7ff5340ffba0 2023-07-14T11:59:18.662 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.677+0000 7ff5237fe700 1 -- 172.21.15.17:0/3701944072 <== mon.2 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff52400fc70 con 0x7ff5340ffba0 2023-07-14T11:59:18.663 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.678+0000 7ff5237fe700 1 --2- 172.21.15.17:0/3701944072 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7ff51c0a1150 0x7ff51c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:18.663 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.678+0000 7ff5237fe700 1 -- 172.21.15.17:0/3701944072 <== mon.2 v2:172.21.15.93:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff5240cdd40 con 0x7ff5340ffba0 2023-07-14T11:59:18.663 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.678+0000 7ff531d9b700 1 --2- 172.21.15.17:0/3701944072 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7ff51c0a1150 0x7ff51c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:18.664 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.679+0000 7ff531d9b700 1 --2- 172.21.15.17:0/3701944072 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7ff51c0a1150 0x7ff51c0a3600 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff528009cb0 tx=0x7ff528009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:18.664 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.680+0000 7ff5237fe700 1 -- 172.21.15.17:0/3701944072 <== mon.2 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff524097920 con 0x7ff5340ffba0 2023-07-14T11:59:18.918 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.937+0000 7ff5389ff700 1 -- 172.21.15.17:0/3701944072 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff5341a0610 con 0x7ff5340ffba0 2023-07-14T11:59:18.919 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.938+0000 7ff5237fe700 1 -- 172.21.15.17:0/3701944072 <== mon.2 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3513 (secure 0 0 0) 0x7ff52402fdd0 con 0x7ff5340ffba0 2023-07-14T11:59:18.919 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:59:18.919 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/55384193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":55384193},{"type":"v1","addr":"172.21.15.17:6835","nonce":55384193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/424956554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":424956554},{"type":"v1","addr":"172.21.15.17:6837","nonce":424956554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/326319712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":326319712},{"type":"v1","addr":"172.21.15.93:6833","nonce":326319712}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/878913679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":878913679},{"type":"v1","addr":"172.21.15.97:6833","nonce":878913679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-07-14T11:59:18.922 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.941+0000 7ff5389ff700 1 -- 172.21.15.17:0/3701944072 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7ff51c0a1150 msgr2=0x7ff51c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:18.922 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.941+0000 7ff5389ff700 1 --2- 172.21.15.17:0/3701944072 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7ff51c0a1150 0x7ff51c0a3600 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff528009cb0 tx=0x7ff528009260 comp rx=0 tx=0).stop 2023-07-14T11:59:18.922 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.941+0000 7ff5389ff700 1 -- 172.21.15.17:0/3701944072 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff5340ffba0 msgr2=0x7ff534197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:18.922 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.941+0000 7ff5389ff700 1 --2- 172.21.15.17:0/3701944072 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff5340ffba0 0x7ff534197b70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff524004730 tx=0x7ff524004760 comp rx=0 tx=0).stop 2023-07-14T11:59:18.922 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.941+0000 7ff5389ff700 1 -- 172.21.15.17:0/3701944072 shutdown_connections 2023-07-14T11:59:18.922 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.941+0000 7ff5389ff700 1 --2- 172.21.15.17:0/3701944072 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7ff51c0a1150 0x7ff51c0a3600 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:18.923 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.941+0000 7ff5389ff700 1 --2- 172.21.15.17:0/3701944072 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff5340ffba0 0x7ff534197b70 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:18.923 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.941+0000 7ff5389ff700 1 --2- 172.21.15.17:0/3701944072 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5341017e0 0x7ff53419f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:18.923 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.941+0000 7ff5389ff700 1 --2- 172.21.15.17:0/3701944072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff534100e30 0x7ff5341980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:18.923 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.941+0000 7ff5389ff700 1 -- 172.21.15.17:0/3701944072 >> 172.21.15.17:0/3701944072 conn(0x7ff5340fb1c0 msgr2=0x7ff5340fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:18.923 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.941+0000 7ff5389ff700 1 -- 172.21.15.17:0/3701944072 shutdown_connections 2023-07-14T11:59:18.923 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:18.941+0000 7ff5389ff700 1 -- 172.21.15.17:0/3701944072 wait complete. 2023-07-14T11:59:18.924 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 5 2023-07-14T11:59:18.970 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:18 smithi093 ceph-mon[128922]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:18.970 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:18 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:18.971 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:18 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:18.971 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:18 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:18.971 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:18 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:59:18.971 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:18 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:59:18.971 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:18 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:59:19.384 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:18 smithi097 ceph-mon[129313]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:19.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:18 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:19.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:18 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:19.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:18 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:19.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:18 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:59:19.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:18 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:59:19.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:18 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:59:19.394 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:18 smithi017 ceph-mon[126339]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:19.394 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:18 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:19.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:18 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:19.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:18 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:19.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:18 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:59:19.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:18 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:59:19.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:18 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:59:19.861 INFO:journalctl@ceph.mds.e.smithi093.stdout:Jul 14 11:59:19 smithi093 systemd[1]: Starting Ceph mds.e for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T11:59:20.185 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:19 smithi093 ceph-mon[128922]: Deploying daemon mds.e on smithi093 2023-07-14T11:59:20.185 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:19 smithi093 ceph-mon[128922]: mds.? [v2:172.21.15.17:6836/424956554,v1:172.21.15.17:6837/424956554] up:boot 2023-07-14T11:59:20.185 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:19 smithi093 ceph-mon[128922]: fsmap 4 up:standby 2023-07-14T11:59:20.185 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:19 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-14T11:59:20.185 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:19 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/3701944072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:59:20.186 INFO:journalctl@ceph.mds.e.smithi093.stdout:Jul 14 11:59:19 smithi093 podman[153774]: 2023-07-14T11:59:20.186 INFO:journalctl@ceph.mds.e.smithi093.stdout:Jul 14 11:59:20 smithi093 ceph-mds[153814]: -- 172.21.15.93:0/2444233549 <== mon.2 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a5976fb1e0 con 0x55a5976bc800 2023-07-14T11:59:20.186 INFO:journalctl@ceph.mds.e.smithi093.stdout:Jul 14 11:59:20 smithi093 ceph-mds[153814]: -- 172.21.15.93:0/2444233549 <== mon.2 v2:172.21.15.93:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a5976e71e0 con 0x55a5976bc800 2023-07-14T11:59:20.186 INFO:journalctl@ceph.mds.e.smithi093.stdout:Jul 14 11:59:20 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-e[153790]: starting mds.e at 2023-07-14T11:59:20.186 INFO:journalctl@ceph.mds.e.smithi093.stdout:Jul 14 11:59:20 smithi093 bash[153774]: 0638049837c04fb09ddf963b1149ed7d3b7fc0666213348fafdd6fc4bbba85ca 2023-07-14T11:59:20.384 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:19 smithi097 ceph-mon[129313]: Deploying daemon mds.e on smithi093 2023-07-14T11:59:20.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:19 smithi097 ceph-mon[129313]: mds.? [v2:172.21.15.17:6836/424956554,v1:172.21.15.17:6837/424956554] up:boot 2023-07-14T11:59:20.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:19 smithi097 ceph-mon[129313]: fsmap 4 up:standby 2023-07-14T11:59:20.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:19 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-14T11:59:20.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:19 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/3701944072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:59:20.394 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:19 smithi017 ceph-mon[126339]: Deploying daemon mds.e on smithi093 2023-07-14T11:59:20.394 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:19 smithi017 ceph-mon[126339]: mds.? [v2:172.21.15.17:6836/424956554,v1:172.21.15.17:6837/424956554] up:boot 2023-07-14T11:59:20.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:19 smithi017 ceph-mon[126339]: fsmap 4 up:standby 2023-07-14T11:59:20.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:19 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-14T11:59:20.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:19 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/3701944072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:59:20.611 INFO:journalctl@ceph.mds.e.smithi093.stdout:Jul 14 11:59:20 smithi093 systemd[1]: Started Ceph mds.e for b3c386cc-223c-11ee-9b33-001a4aab830c. 2023-07-14T11:59:20.942 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:59:21.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:20 smithi097 ceph-mon[129313]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:21.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:20 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:21.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:20 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:21.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:20 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:21.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:20 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:59:21.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:20 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:59:21.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:20 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:59:21.155 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.174+0000 7f198a1ff700 1 -- 172.21.15.17:0/2433444401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19840ffba0 msgr2=0x7f19840fffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:21.155 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.174+0000 7f198a1ff700 1 --2- 172.21.15.17:0/2433444401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19840ffba0 0x7f19840fffb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1974009990 tx=0x7f1974009ca0 comp rx=0 tx=0).stop 2023-07-14T11:59:21.155 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.174+0000 7f198a1ff700 1 -- 172.21.15.17:0/2433444401 shutdown_connections 2023-07-14T11:59:21.155 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.174+0000 7f198a1ff700 1 --2- 172.21.15.17:0/2433444401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19841017e0 0x7f1984108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:21.155 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.174+0000 7f198a1ff700 1 --2- 172.21.15.17:0/2433444401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19840ffba0 0x7f19840fffb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:21.156 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.174+0000 7f198a1ff700 1 --2- 172.21.15.17:0/2433444401 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1984100e30 0x7f19841012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:21.156 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.174+0000 7f198a1ff700 1 -- 172.21.15.17:0/2433444401 >> 172.21.15.17:0/2433444401 conn(0x7f19840fb1c0 msgr2=0x7f19840fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:21.156 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.174+0000 7f198a1ff700 1 -- 172.21.15.17:0/2433444401 shutdown_connections 2023-07-14T11:59:21.156 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.174+0000 7f198a1ff700 1 -- 172.21.15.17:0/2433444401 wait complete. 2023-07-14T11:59:21.158 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.175+0000 7f198a1ff700 1 Processor -- start 2023-07-14T11:59:21.158 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.175+0000 7f198a1ff700 1 -- start start 2023-07-14T11:59:21.158 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.175+0000 7f198a1ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f19840ffba0 0x7f19840742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:21.158 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.175+0000 7f198a1ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1984100e30 0x7f1984072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:21.158 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.176+0000 7f198a1ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19841017e0 0x7f1984072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:21.158 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.176+0000 7f198a1ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19840735e0 con 0x7f19841017e0 2023-07-14T11:59:21.159 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.176+0000 7f198a1ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1984073750 con 0x7f19840ffba0 2023-07-14T11:59:21.159 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.176+0000 7f198a1ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198419f940 con 0x7f1984100e30 2023-07-14T11:59:21.159 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.176+0000 7f1982ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1984100e30 0x7f1984072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:21.159 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.176+0000 7f1983fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19841017e0 0x7f1984072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:21.159 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.176+0000 7f19837fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f19840ffba0 0x7f19840742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:21.159 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.176+0000 7f1982ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1984100e30 0x7f1984072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.17:47268/0 (socket says 172.21.15.17:47268) 2023-07-14T11:59:21.159 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.176+0000 7f1982ffd700 1 -- 172.21.15.17:0/2325359991 learned_addr learned my addr 172.21.15.17:0/2325359991 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:59:21.159 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.176+0000 7f1982ffd700 1 -- 172.21.15.17:0/2325359991 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f19840ffba0 msgr2=0x7f19840742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:21.160 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.176+0000 7f1982ffd700 1 --2- 172.21.15.17:0/2325359991 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f19840ffba0 0x7f19840742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:21.160 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.176+0000 7f1982ffd700 1 -- 172.21.15.17:0/2325359991 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19841017e0 msgr2=0x7f1984072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:21.160 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.176+0000 7f1982ffd700 1 --2- 172.21.15.17:0/2325359991 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19841017e0 0x7f1984072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:21.160 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.176+0000 7f1982ffd700 1 -- 172.21.15.17:0/2325359991 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19740095f0 con 0x7f1984100e30 2023-07-14T11:59:21.160 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.176+0000 7f1982ffd700 1 --2- 172.21.15.17:0/2325359991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1984100e30 0x7f1984072900 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f197800e9a0 tx=0x7f197800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:21.160 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.177+0000 7f1980ff9700 1 -- 172.21.15.17:0/2325359991 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1978009790 con 0x7f1984100e30 2023-07-14T11:59:21.160 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.177+0000 7f1980ff9700 1 -- 172.21.15.17:0/2325359991 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1978010430 con 0x7f1984100e30 2023-07-14T11:59:21.161 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.177+0000 7f198a1ff700 1 -- 172.21.15.17:0/2325359991 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f198419fc20 con 0x7f1984100e30 2023-07-14T11:59:21.161 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.177+0000 7f1980ff9700 1 -- 172.21.15.17:0/2325359991 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1978018680 con 0x7f1984100e30 2023-07-14T11:59:21.161 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.177+0000 7f198a1ff700 1 -- 172.21.15.17:0/2325359991 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19841a0130 con 0x7f1984100e30 2023-07-14T11:59:21.161 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.178+0000 7f198a1ff700 1 -- 172.21.15.17:0/2325359991 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f198410b3c0 con 0x7f1984100e30 2023-07-14T11:59:21.162 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.180+0000 7f1980ff9700 1 -- 172.21.15.17:0/2325359991 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f19780105a0 con 0x7f1984100e30 2023-07-14T11:59:21.162 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.181+0000 7f1980ff9700 1 --2- 172.21.15.17:0/2325359991 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f196c0a1150 0x7f196c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:21.162 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.181+0000 7f1980ff9700 1 -- 172.21.15.17:0/2325359991 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1978014070 con 0x7f1984100e30 2023-07-14T11:59:21.163 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.181+0000 7f19837fe700 1 --2- 172.21.15.17:0/2325359991 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f196c0a1150 0x7f196c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:21.163 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.182+0000 7f19837fe700 1 --2- 172.21.15.17:0/2325359991 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f196c0a1150 0x7f196c0a3600 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f197400b2e0 tx=0x7f1974005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:21.164 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.183+0000 7f1980ff9700 1 -- 172.21.15.17:0/2325359991 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1978096190 con 0x7f1984100e30 2023-07-14T11:59:21.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:20 smithi093 ceph-mon[128922]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:21.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:20 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:21.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:20 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:21.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:20 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:21.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:20 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:59:21.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:20 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:59:21.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:20 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:59:21.394 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:20 smithi017 ceph-mon[126339]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:21.394 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:20 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:21.394 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:20 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:21.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:20 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:21.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:20 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:59:21.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:20 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:59:21.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:20 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:59:21.424 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.443+0000 7f198a1ff700 1 -- 172.21.15.17:0/2325359991 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1984106b20 con 0x7f1984100e30 2023-07-14T11:59:21.424 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.444+0000 7f1980ff9700 1 -- 172.21.15.17:0/2325359991 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4270 (secure 0 0 0) 0x7f19780997b0 con 0x7f1984100e30 2023-07-14T11:59:21.425 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:59:21.425 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/55384193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":55384193},{"type":"v1","addr":"172.21.15.17:6835","nonce":55384193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/424956554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":424956554},{"type":"v1","addr":"172.21.15.17:6837","nonce":424956554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/326319712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":326319712},{"type":"v1","addr":"172.21.15.93:6833","nonce":326319712}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/878913679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":878913679},{"type":"v1","addr":"172.21.15.97:6833","nonce":878913679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/2779420503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":2779420503},{"type":"v1","addr":"172.21.15.93:6835","nonce":2779420503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2023-07-14T11:59:21.427 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.446+0000 7f198a1ff700 1 -- 172.21.15.17:0/2325359991 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f196c0a1150 msgr2=0x7f196c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:21.427 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.446+0000 7f198a1ff700 1 --2- 172.21.15.17:0/2325359991 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f196c0a1150 0x7f196c0a3600 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f197400b2e0 tx=0x7f1974005fb0 comp rx=0 tx=0).stop 2023-07-14T11:59:21.427 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.446+0000 7f198a1ff700 1 -- 172.21.15.17:0/2325359991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1984100e30 msgr2=0x7f1984072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:21.427 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.446+0000 7f198a1ff700 1 --2- 172.21.15.17:0/2325359991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1984100e30 0x7f1984072900 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f197800e9a0 tx=0x7f197800ecb0 comp rx=0 tx=0).stop 2023-07-14T11:59:21.428 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.446+0000 7f198a1ff700 1 -- 172.21.15.17:0/2325359991 shutdown_connections 2023-07-14T11:59:21.428 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.446+0000 7f198a1ff700 1 --2- 172.21.15.17:0/2325359991 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f196c0a1150 0x7f196c0a3600 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:21.428 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.446+0000 7f198a1ff700 1 --2- 172.21.15.17:0/2325359991 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f19840ffba0 0x7f19840742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:21.428 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.446+0000 7f198a1ff700 1 --2- 172.21.15.17:0/2325359991 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19841017e0 0x7f1984072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:21.428 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.446+0000 7f198a1ff700 1 --2- 172.21.15.17:0/2325359991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1984100e30 0x7f1984072900 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:21.428 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.446+0000 7f198a1ff700 1 -- 172.21.15.17:0/2325359991 >> 172.21.15.17:0/2325359991 conn(0x7f19840fb1c0 msgr2=0x7f19840fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:21.429 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.446+0000 7f198a1ff700 1 -- 172.21.15.17:0/2325359991 shutdown_connections 2023-07-14T11:59:21.429 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:21.446+0000 7f198a1ff700 1 -- 172.21.15.17:0/2325359991 wait complete. 2023-07-14T11:59:21.429 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 6 2023-07-14T11:59:21.885 INFO:journalctl@ceph.mds.f.smithi097.stdout:Jul 14 11:59:21 smithi097 systemd[1]: Starting Ceph mds.f for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T11:59:22.241 INFO:journalctl@ceph.mds.f.smithi097.stdout:Jul 14 11:59:21 smithi097 podman[154979]: 2023-07-14T11:59:22.242 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:21 smithi097 ceph-mon[129313]: Deploying daemon mds.f on smithi097 2023-07-14T11:59:22.242 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:21 smithi097 ceph-mon[129313]: mds.? [v2:172.21.15.93:6834/2779420503,v1:172.21.15.93:6835/2779420503] up:boot 2023-07-14T11:59:22.242 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:21 smithi097 ceph-mon[129313]: fsmap 5 up:standby 2023-07-14T11:59:22.242 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:21 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-14T11:59:22.242 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:21 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/2325359991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:59:22.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:21 smithi093 ceph-mon[128922]: Deploying daemon mds.f on smithi097 2023-07-14T11:59:22.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:21 smithi093 ceph-mon[128922]: mds.? [v2:172.21.15.93:6834/2779420503,v1:172.21.15.93:6835/2779420503] up:boot 2023-07-14T11:59:22.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:21 smithi093 ceph-mon[128922]: fsmap 5 up:standby 2023-07-14T11:59:22.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:21 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-14T11:59:22.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:21 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/2325359991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:59:22.394 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:21 smithi017 ceph-mon[126339]: Deploying daemon mds.f on smithi097 2023-07-14T11:59:22.394 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:21 smithi017 ceph-mon[126339]: mds.? [v2:172.21.15.93:6834/2779420503,v1:172.21.15.93:6835/2779420503] up:boot 2023-07-14T11:59:22.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:21 smithi017 ceph-mon[126339]: fsmap 5 up:standby 2023-07-14T11:59:22.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:21 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-14T11:59:22.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:21 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/2325359991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:59:22.634 INFO:journalctl@ceph.mds.f.smithi097.stdout:Jul 14 11:59:22 smithi097 ceph-mds[155019]: -- 172.21.15.97:0/3052117076 <== mon.1 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5616b2d9f1e0 con 0x5616b2d75000 2023-07-14T11:59:22.635 INFO:journalctl@ceph.mds.f.smithi097.stdout:Jul 14 11:59:22 smithi097 bash[154979]: 6df1162247efbf6ec78c022450452762b4b9f243be350445001d1b3366bb747d 2023-07-14T11:59:22.635 INFO:journalctl@ceph.mds.f.smithi097.stdout:Jul 14 11:59:22 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-f[154995]: starting mds.f at 2023-07-14T11:59:22.635 INFO:journalctl@ceph.mds.f.smithi097.stdout:Jul 14 11:59:22 smithi097 systemd[1]: Started Ceph mds.f for b3c386cc-223c-11ee-9b33-001a4aab830c. 2023-07-14T11:59:23.286 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:22 smithi097 ceph-mon[129313]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:23.286 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:22 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:23.286 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:22 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:23.286 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:22 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:23.286 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:22 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:59:23.286 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:22 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:59:23.287 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:22 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:59:23.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:22 smithi093 ceph-mon[128922]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:23.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:22 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:23.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:22 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:23.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:22 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:23.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:22 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:59:23.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:22 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:59:23.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:22 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:59:23.394 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:22 smithi017 ceph-mon[126339]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:23.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:22 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:23.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:22 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:23.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:22 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:23.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:22 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:59:23.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:22 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:59:23.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:22 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:59:23.447 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:59:23.662 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.680+0000 7f7e54bff700 1 -- 172.21.15.17:0/2399658133 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7e500ffba0 msgr2=0x7f7e500fffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:23.662 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.680+0000 7f7e54bff700 1 --2- 172.21.15.17:0/2399658133 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7e500ffba0 0x7f7e500fffb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7e40009990 tx=0x7f7e40009ca0 comp rx=0 tx=0).stop 2023-07-14T11:59:23.662 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.680+0000 7f7e54bff700 1 -- 172.21.15.17:0/2399658133 shutdown_connections 2023-07-14T11:59:23.663 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.680+0000 7f7e54bff700 1 --2- 172.21.15.17:0/2399658133 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7e501017e0 0x7f7e50108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:23.663 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.680+0000 7f7e54bff700 1 --2- 172.21.15.17:0/2399658133 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7e500ffba0 0x7f7e500fffb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:23.663 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.680+0000 7f7e54bff700 1 --2- 172.21.15.17:0/2399658133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e50100e30 0x7f7e501012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:23.663 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.681+0000 7f7e54bff700 1 -- 172.21.15.17:0/2399658133 >> 172.21.15.17:0/2399658133 conn(0x7f7e500fb1c0 msgr2=0x7f7e500fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:23.663 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.681+0000 7f7e54bff700 1 -- 172.21.15.17:0/2399658133 shutdown_connections 2023-07-14T11:59:23.664 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.681+0000 7f7e54bff700 1 -- 172.21.15.17:0/2399658133 wait complete. 2023-07-14T11:59:23.666 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.682+0000 7f7e54bff700 1 Processor -- start 2023-07-14T11:59:23.667 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.682+0000 7f7e54bff700 1 -- start start 2023-07-14T11:59:23.667 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.682+0000 7f7e54bff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7e500ffba0 0x7f7e500742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:23.667 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.682+0000 7f7e54bff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7e50100e30 0x7f7e50072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:23.667 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.682+0000 7f7e54bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e501017e0 0x7f7e50072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:23.667 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.682+0000 7f7e54bff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e500735e0 con 0x7f7e50100e30 2023-07-14T11:59:23.668 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.682+0000 7f7e54bff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e50073750 con 0x7f7e500ffba0 2023-07-14T11:59:23.668 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.682+0000 7f7e54bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5019f940 con 0x7f7e501017e0 2023-07-14T11:59:23.668 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.682+0000 7f7e4dd9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7e50100e30 0x7f7e50072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:23.668 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.683+0000 7f7e4dd9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7e50100e30 0x7f7e50072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56000/0 (socket says 172.21.15.17:56000) 2023-07-14T11:59:23.669 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.683+0000 7f7e4dd9b700 1 -- 172.21.15.17:0/3026485356 learned_addr learned my addr 172.21.15.17:0/3026485356 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:59:23.669 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.683+0000 7f7e4e59c700 1 --2- 172.21.15.17:0/3026485356 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7e500ffba0 0x7f7e500742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:23.669 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.683+0000 7f7e4ed9d700 1 --2- 172.21.15.17:0/3026485356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e501017e0 0x7f7e50072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:23.669 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.683+0000 7f7e4e59c700 1 -- 172.21.15.17:0/3026485356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e501017e0 msgr2=0x7f7e50072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:23.669 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.683+0000 7f7e4e59c700 1 --2- 172.21.15.17:0/3026485356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e501017e0 0x7f7e50072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:23.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.683+0000 7f7e4e59c700 1 -- 172.21.15.17:0/3026485356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7e50100e30 msgr2=0x7f7e50072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:23.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.683+0000 7f7e4e59c700 1 --2- 172.21.15.17:0/3026485356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7e50100e30 0x7f7e50072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:23.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.683+0000 7f7e4e59c700 1 -- 172.21.15.17:0/3026485356 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e400095f0 con 0x7f7e500ffba0 2023-07-14T11:59:23.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.683+0000 7f7e4dd9b700 1 --2- 172.21.15.17:0/3026485356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7e50100e30 0x7f7e50072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:59:23.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.683+0000 7f7e4e59c700 1 --2- 172.21.15.17:0/3026485356 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7e500ffba0 0x7f7e500742c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7e40000c00 tx=0x7f7e4000f670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:23.671 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.683+0000 7f7e3f7fe700 1 -- 172.21.15.17:0/3026485356 <== mon.2 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4001d070 con 0x7f7e500ffba0 2023-07-14T11:59:23.671 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.683+0000 7f7e3f7fe700 1 -- 172.21.15.17:0/3026485356 <== mon.2 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e40005050 con 0x7f7e500ffba0 2023-07-14T11:59:23.671 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.683+0000 7f7e54bff700 1 -- 172.21.15.17:0/3026485356 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e5019fbc0 con 0x7f7e500ffba0 2023-07-14T11:59:23.671 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.683+0000 7f7e3f7fe700 1 -- 172.21.15.17:0/3026485356 <== mon.2 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e40004a70 con 0x7f7e500ffba0 2023-07-14T11:59:23.672 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.684+0000 7f7e54bff700 1 -- 172.21.15.17:0/3026485356 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e5019ffe0 con 0x7f7e500ffba0 2023-07-14T11:59:23.672 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.684+0000 7f7e54bff700 1 -- 172.21.15.17:0/3026485356 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e50106b90 con 0x7f7e500ffba0 2023-07-14T11:59:23.673 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.687+0000 7f7e3f7fe700 1 -- 172.21.15.17:0/3026485356 <== mon.2 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7e400051c0 con 0x7f7e500ffba0 2023-07-14T11:59:23.673 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.688+0000 7f7e3f7fe700 1 --2- 172.21.15.17:0/3026485356 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f7e380a1150 0x7f7e380a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:23.673 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.688+0000 7f7e3f7fe700 1 -- 172.21.15.17:0/3026485356 <== mon.2 v2:172.21.15.93:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7e400cd880 con 0x7f7e500ffba0 2023-07-14T11:59:23.673 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.688+0000 7f7e4dd9b700 1 --2- 172.21.15.17:0/3026485356 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f7e380a1150 0x7f7e380a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:23.674 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.689+0000 7f7e4dd9b700 1 --2- 172.21.15.17:0/3026485356 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f7e380a1150 0x7f7e380a3600 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7e44005950 tx=0x7f7e4400b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:23.674 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.690+0000 7f7e3f7fe700 1 -- 172.21.15.17:0/3026485356 <== mon.2 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7e40097460 con 0x7f7e500ffba0 2023-07-14T11:59:23.923 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.942+0000 7f7e54bff700 1 -- 172.21.15.17:0/3026485356 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e5002c850 con 0x7f7e500ffba0 2023-07-14T11:59:23.924 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.943+0000 7f7e3f7fe700 1 -- 172.21.15.17:0/3026485356 <== mon.2 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5027 (secure 0 0 0) 0x7f7e4009aa80 con 0x7f7e500ffba0 2023-07-14T11:59:23.924 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:59:23.925 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/55384193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":55384193},{"type":"v1","addr":"172.21.15.17:6835","nonce":55384193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/424956554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":424956554},{"type":"v1","addr":"172.21.15.17:6837","nonce":424956554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/326319712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":326319712},{"type":"v1","addr":"172.21.15.93:6833","nonce":326319712}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/878913679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":878913679},{"type":"v1","addr":"172.21.15.97:6833","nonce":878913679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/2779420503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":2779420503},{"type":"v1","addr":"172.21.15.93:6835","nonce":2779420503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2740799393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2740799393},{"type":"v1","addr":"172.21.15.97:6835","nonce":2740799393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[]} 2023-07-14T11:59:23.927 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.946+0000 7f7e54bff700 1 -- 172.21.15.17:0/3026485356 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f7e380a1150 msgr2=0x7f7e380a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:23.927 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.946+0000 7f7e54bff700 1 --2- 172.21.15.17:0/3026485356 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f7e380a1150 0x7f7e380a3600 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7e44005950 tx=0x7f7e4400b500 comp rx=0 tx=0).stop 2023-07-14T11:59:23.927 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.946+0000 7f7e54bff700 1 -- 172.21.15.17:0/3026485356 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7e500ffba0 msgr2=0x7f7e500742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:23.927 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.946+0000 7f7e54bff700 1 --2- 172.21.15.17:0/3026485356 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7e500ffba0 0x7f7e500742c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7e40000c00 tx=0x7f7e4000f670 comp rx=0 tx=0).stop 2023-07-14T11:59:23.927 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.946+0000 7f7e54bff700 1 -- 172.21.15.17:0/3026485356 shutdown_connections 2023-07-14T11:59:23.927 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.946+0000 7f7e54bff700 1 --2- 172.21.15.17:0/3026485356 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f7e380a1150 0x7f7e380a3600 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:23.928 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.946+0000 7f7e54bff700 1 --2- 172.21.15.17:0/3026485356 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7e500ffba0 0x7f7e500742c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:23.928 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.946+0000 7f7e54bff700 1 --2- 172.21.15.17:0/3026485356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e501017e0 0x7f7e50072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:23.928 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.946+0000 7f7e54bff700 1 --2- 172.21.15.17:0/3026485356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7e50100e30 0x7f7e50072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:23.928 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.946+0000 7f7e54bff700 1 -- 172.21.15.17:0/3026485356 >> 172.21.15.17:0/3026485356 conn(0x7f7e500fb1c0 msgr2=0x7f7e500fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:23.928 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.946+0000 7f7e54bff700 1 -- 172.21.15.17:0/3026485356 shutdown_connections 2023-07-14T11:59:23.928 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:23.946+0000 7f7e54bff700 1 -- 172.21.15.17:0/3026485356 wait complete. 2023-07-14T11:59:23.929 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 7 2023-07-14T11:59:23.945 INFO:journalctl@ceph.mds.c.smithi097.stdout:Jul 14 11:59:23 smithi097 systemd[1]: Starting Ceph mds.c for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T11:59:24.302 INFO:journalctl@ceph.mds.c.smithi097.stdout:Jul 14 11:59:24 smithi097 podman[155316]: 2023-07-14T11:59:24.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:23 smithi097 ceph-mon[129313]: Deploying daemon mds.c on smithi097 2023-07-14T11:59:24.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:23 smithi097 ceph-mon[129313]: mds.? [v2:172.21.15.97:6834/2740799393,v1:172.21.15.97:6835/2740799393] up:boot 2023-07-14T11:59:24.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:23 smithi097 ceph-mon[129313]: fsmap 6 up:standby 2023-07-14T11:59:24.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:23 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-14T11:59:24.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:23 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/3026485356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:59:24.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:23 smithi093 ceph-mon[128922]: Deploying daemon mds.c on smithi097 2023-07-14T11:59:24.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:23 smithi093 ceph-mon[128922]: mds.? [v2:172.21.15.97:6834/2740799393,v1:172.21.15.97:6835/2740799393] up:boot 2023-07-14T11:59:24.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:23 smithi093 ceph-mon[128922]: fsmap 6 up:standby 2023-07-14T11:59:24.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:23 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-14T11:59:24.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:23 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/3026485356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:59:24.394 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:23 smithi017 ceph-mon[126339]: Deploying daemon mds.c on smithi097 2023-07-14T11:59:24.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:23 smithi017 ceph-mon[126339]: mds.? [v2:172.21.15.97:6834/2740799393,v1:172.21.15.97:6835/2740799393] up:boot 2023-07-14T11:59:24.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:23 smithi017 ceph-mon[126339]: fsmap 6 up:standby 2023-07-14T11:59:24.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:23 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-14T11:59:24.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:23 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/3026485356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:59:24.885 INFO:journalctl@ceph.mds.c.smithi097.stdout:Jul 14 11:59:24 smithi097 ceph-mds[155357]: -- 172.21.15.97:0/283358612 <== mon.1 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a6c0fc91e0 con 0x55a6c0f9e800 2023-07-14T11:59:24.886 INFO:journalctl@ceph.mds.c.smithi097.stdout:Jul 14 11:59:24 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-c[155333]: starting mds.c at 2023-07-14T11:59:24.886 INFO:journalctl@ceph.mds.c.smithi097.stdout:Jul 14 11:59:24 smithi097 bash[155316]: e18102a30fe4bee63504512d811ce0aa2ca08a8de961ef9b5a133266e7003420 2023-07-14T11:59:24.886 INFO:journalctl@ceph.mds.c.smithi097.stdout:Jul 14 11:59:24 smithi097 systemd[1]: Started Ceph mds.c for b3c386cc-223c-11ee-9b33-001a4aab830c. 2023-07-14T11:59:25.217 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:24 smithi093 ceph-mon[128922]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:25.217 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:24 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:25.217 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:24 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:25.217 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:24 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:25.218 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:24 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:59:25.218 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:24 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:59:25.218 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:24 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:59:25.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:24 smithi097 ceph-mon[129313]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:25.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:24 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:25.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:24 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:25.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:24 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:25.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:24 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:59:25.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:24 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:59:25.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:24 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:59:25.394 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:24 smithi017 ceph-mon[126339]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:25.394 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:24 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:25.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:24 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:25.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:24 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:25.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:24 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:59:25.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:24 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:59:25.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:24 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:59:25.947 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:59:26.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:25 smithi093 ceph-mon[128922]: Deploying daemon mds.h on smithi093 2023-07-14T11:59:26.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:25 smithi093 ceph-mon[128922]: mds.? [v2:172.21.15.97:6836/3684187131,v1:172.21.15.97:6837/3684187131] up:boot 2023-07-14T11:59:26.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:25 smithi093 ceph-mon[128922]: fsmap 7 up:standby 2023-07-14T11:59:26.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:25 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-14T11:59:26.112 INFO:journalctl@ceph.mds.h.smithi093.stdout:Jul 14 11:59:26 smithi093 systemd[1]: Starting Ceph mds.h for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T11:59:26.162 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.181+0000 7fb0ae7ff700 1 -- 172.21.15.17:0/2012849203 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0a80744b0 msgr2=0x7fb0a80729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:26.162 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.181+0000 7fb0ae7ff700 1 --2- 172.21.15.17:0/2012849203 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0a80744b0 0x7fb0a80729a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb098009990 tx=0x7fb098009ca0 comp rx=0 tx=0).stop 2023-07-14T11:59:26.163 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.181+0000 7fb0ae7ff700 1 -- 172.21.15.17:0/2012849203 shutdown_connections 2023-07-14T11:59:26.163 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.181+0000 7fb0ae7ff700 1 --2- 172.21.15.17:0/2012849203 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0a80744b0 0x7fb0a80729a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:26.163 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.181+0000 7fb0ae7ff700 1 --2- 172.21.15.17:0/2012849203 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0a8107a20 0x7fb0a8109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:26.163 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.181+0000 7fb0ae7ff700 1 --2- 172.21.15.17:0/2012849203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0a8072ee0 0x7fb0a8073350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:26.163 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.181+0000 7fb0ae7ff700 1 -- 172.21.15.17:0/2012849203 >> 172.21.15.17:0/2012849203 conn(0x7fb0a80fb1c0 msgr2=0x7fb0a80fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:26.163 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.181+0000 7fb0ae7ff700 1 -- 172.21.15.17:0/2012849203 shutdown_connections 2023-07-14T11:59:26.163 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.181+0000 7fb0ae7ff700 1 -- 172.21.15.17:0/2012849203 wait complete. 2023-07-14T11:59:26.165 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.182+0000 7fb0ae7ff700 1 Processor -- start 2023-07-14T11:59:26.165 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.183+0000 7fb0ae7ff700 1 -- start start 2023-07-14T11:59:26.165 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.183+0000 7fb0ae7ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0a8072ee0 0x7fb0a8197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:26.165 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.183+0000 7fb0ae7ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0a80744b0 0x7fb0a8198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:26.166 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.183+0000 7fb0ae7ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0a8107a20 0x7fb0a819f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:26.166 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.183+0000 7fb0ae7ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a819f6c0 con 0x7fb0a8107a20 2023-07-14T11:59:26.166 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.183+0000 7fb0ae7ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a819f800 con 0x7fb0a8072ee0 2023-07-14T11:59:26.166 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.183+0000 7fb0ae7ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a819f990 con 0x7fb0a80744b0 2023-07-14T11:59:26.166 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.183+0000 7fb0a7fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0a8072ee0 0x7fb0a8197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:26.166 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.183+0000 7fb0a7fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0a8072ee0 0x7fb0a8197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.17:54022/0 (socket says 172.21.15.17:54022) 2023-07-14T11:59:26.167 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.183+0000 7fb0a7fff700 1 -- 172.21.15.17:0/2735021403 learned_addr learned my addr 172.21.15.17:0/2735021403 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:59:26.167 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.183+0000 7fb0acd9c700 1 --2- 172.21.15.17:0/2735021403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0a8107a20 0x7fb0a819f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:26.167 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.183+0000 7fb0a77fe700 1 --2- 172.21.15.17:0/2735021403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0a80744b0 0x7fb0a8198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:26.167 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.184+0000 7fb0a7fff700 1 -- 172.21.15.17:0/2735021403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0a80744b0 msgr2=0x7fb0a8198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:26.167 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.184+0000 7fb0a7fff700 1 --2- 172.21.15.17:0/2735021403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0a80744b0 0x7fb0a8198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:26.167 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.184+0000 7fb0a7fff700 1 -- 172.21.15.17:0/2735021403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0a8107a20 msgr2=0x7fb0a819f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:26.167 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.184+0000 7fb0a7fff700 1 --2- 172.21.15.17:0/2735021403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0a8107a20 0x7fb0a819f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:26.168 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.184+0000 7fb0a7fff700 1 -- 172.21.15.17:0/2735021403 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a0009c40 con 0x7fb0a8072ee0 2023-07-14T11:59:26.168 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.184+0000 7fb0acd9c700 1 --2- 172.21.15.17:0/2735021403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0a8107a20 0x7fb0a819f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:59:26.168 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.184+0000 7fb0a7fff700 1 --2- 172.21.15.17:0/2735021403 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0a8072ee0 0x7fb0a8197b40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb098004710 tx=0x7fb098009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:26.168 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.184+0000 7fb0a57fa700 1 -- 172.21.15.17:0/2735021403 <== mon.2 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09801d070 con 0x7fb0a8072ee0 2023-07-14T11:59:26.168 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.184+0000 7fb0ae7ff700 1 -- 172.21.15.17:0/2735021403 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0980095f0 con 0x7fb0a8072ee0 2023-07-14T11:59:26.168 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.184+0000 7fb0a57fa700 1 -- 172.21.15.17:0/2735021403 <== mon.2 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb098021410 con 0x7fb0a8072ee0 2023-07-14T11:59:26.168 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.184+0000 7fb0a57fa700 1 -- 172.21.15.17:0/2735021403 <== mon.2 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb098017650 con 0x7fb0a8072ee0 2023-07-14T11:59:26.168 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.184+0000 7fb0ae7ff700 1 -- 172.21.15.17:0/2735021403 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0a819fea0 con 0x7fb0a8072ee0 2023-07-14T11:59:26.169 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.185+0000 7fb0ae7ff700 1 -- 172.21.15.17:0/2735021403 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0a81061b0 con 0x7fb0a8072ee0 2023-07-14T11:59:26.169 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.187+0000 7fb0a57fa700 1 -- 172.21.15.17:0/2735021403 <== mon.2 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb098021580 con 0x7fb0a8072ee0 2023-07-14T11:59:26.170 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.189+0000 7fb0a57fa700 1 --2- 172.21.15.17:0/2735021403 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fb0900a1150 0x7fb0900a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:26.170 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.189+0000 7fb0a57fa700 1 -- 172.21.15.17:0/2735021403 <== mon.2 v2:172.21.15.93:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb0980cdfc0 con 0x7fb0a8072ee0 2023-07-14T11:59:26.170 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.189+0000 7fb0a77fe700 1 --2- 172.21.15.17:0/2735021403 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fb0900a1150 0x7fb0900a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:26.170 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.189+0000 7fb0a77fe700 1 --2- 172.21.15.17:0/2735021403 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fb0900a1150 0x7fb0900a3600 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb09c009c60 tx=0x7fb09c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:26.171 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.191+0000 7fb0a57fa700 1 -- 172.21.15.17:0/2735021403 <== mon.2 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb098097ba0 con 0x7fb0a8072ee0 2023-07-14T11:59:26.384 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:25 smithi097 ceph-mon[129313]: Deploying daemon mds.h on smithi093 2023-07-14T11:59:26.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:25 smithi097 ceph-mon[129313]: mds.? [v2:172.21.15.97:6836/3684187131,v1:172.21.15.97:6837/3684187131] up:boot 2023-07-14T11:59:26.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:25 smithi097 ceph-mon[129313]: fsmap 7 up:standby 2023-07-14T11:59:26.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:25 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-14T11:59:26.394 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:25 smithi017 ceph-mon[126339]: Deploying daemon mds.h on smithi093 2023-07-14T11:59:26.394 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:25 smithi017 ceph-mon[126339]: mds.? [v2:172.21.15.97:6836/3684187131,v1:172.21.15.97:6837/3684187131] up:boot 2023-07-14T11:59:26.394 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:25 smithi017 ceph-mon[126339]: fsmap 7 up:standby 2023-07-14T11:59:26.394 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:25 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-14T11:59:26.423 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.443+0000 7fb0ae7ff700 1 -- 172.21.15.17:0/2735021403 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb0a802c850 con 0x7fb0a8072ee0 2023-07-14T11:59:26.425 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.444+0000 7fb0a57fa700 1 -- 172.21.15.17:0/2735021403 <== mon.2 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+5784 (secure 0 0 0) 0x7fb09809b1c0 con 0x7fb0a8072ee0 2023-07-14T11:59:26.425 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:59:26.425 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/55384193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":55384193},{"type":"v1","addr":"172.21.15.17:6835","nonce":55384193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/424956554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":424956554},{"type":"v1","addr":"172.21.15.17:6837","nonce":424956554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/326319712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":326319712},{"type":"v1","addr":"172.21.15.93:6833","nonce":326319712}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/878913679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":878913679},{"type":"v1","addr":"172.21.15.97:6833","nonce":878913679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/2779420503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":2779420503},{"type":"v1","addr":"172.21.15.93:6835","nonce":2779420503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2740799393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2740799393},{"type":"v1","addr":"172.21.15.97:6835","nonce":2740799393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3684187131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3684187131},{"type":"v1","addr":"172.21.15.97:6837","nonce":3684187131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2023-07-14T11:59:26.428 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.447+0000 7fb0ae7ff700 1 -- 172.21.15.17:0/2735021403 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fb0900a1150 msgr2=0x7fb0900a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:26.428 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.447+0000 7fb0ae7ff700 1 --2- 172.21.15.17:0/2735021403 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fb0900a1150 0x7fb0900a3600 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb09c009c60 tx=0x7fb09c009260 comp rx=0 tx=0).stop 2023-07-14T11:59:26.428 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.447+0000 7fb0ae7ff700 1 -- 172.21.15.17:0/2735021403 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0a8072ee0 msgr2=0x7fb0a8197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:26.428 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.447+0000 7fb0ae7ff700 1 --2- 172.21.15.17:0/2735021403 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0a8072ee0 0x7fb0a8197b40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb098004710 tx=0x7fb098009580 comp rx=0 tx=0).stop 2023-07-14T11:59:26.428 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.447+0000 7fb0ae7ff700 1 -- 172.21.15.17:0/2735021403 shutdown_connections 2023-07-14T11:59:26.429 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.447+0000 7fb0ae7ff700 1 --2- 172.21.15.17:0/2735021403 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fb0900a1150 0x7fb0900a3600 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:26.429 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.447+0000 7fb0ae7ff700 1 --2- 172.21.15.17:0/2735021403 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0a8072ee0 0x7fb0a8197b40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:26.429 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.447+0000 7fb0ae7ff700 1 --2- 172.21.15.17:0/2735021403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0a8107a20 0x7fb0a819f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:26.429 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.447+0000 7fb0ae7ff700 1 --2- 172.21.15.17:0/2735021403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0a80744b0 0x7fb0a8198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:26.429 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.447+0000 7fb0ae7ff700 1 -- 172.21.15.17:0/2735021403 >> 172.21.15.17:0/2735021403 conn(0x7fb0a80fb1c0 msgr2=0x7fb0a80fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:26.429 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.447+0000 7fb0ae7ff700 1 -- 172.21.15.17:0/2735021403 shutdown_connections 2023-07-14T11:59:26.429 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:26.447+0000 7fb0ae7ff700 1 -- 172.21.15.17:0/2735021403 wait complete. 2023-07-14T11:59:26.430 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 8 2023-07-14T11:59:26.658 INFO:journalctl@ceph.mds.h.smithi093.stdout:Jul 14 11:59:26 smithi093 podman[154125]: 2023-07-14T11:59:26.659 INFO:journalctl@ceph.mds.h.smithi093.stdout:Jul 14 11:59:26 smithi093 ceph-mds[154166]: -- 172.21.15.93:0/3232818653 <== mon.2 v2:172.21.15.93:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560b601d31e0 con 0x560b601a9000 2023-07-14T11:59:26.659 INFO:journalctl@ceph.mds.h.smithi093.stdout:Jul 14 11:59:26 smithi093 bash[154125]: a4010356a2bfbc10ce74aa97fca16986c0f5c43f16a26640f395930a4640ac09 2023-07-14T11:59:26.659 INFO:journalctl@ceph.mds.h.smithi093.stdout:Jul 14 11:59:26 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-h[154141]: starting mds.h at 2023-07-14T11:59:26.949 INFO:journalctl@ceph.mds.h.smithi093.stdout:Jul 14 11:59:26 smithi093 systemd[1]: Started Ceph mds.h for b3c386cc-223c-11ee-9b33-001a4aab830c. 2023-07-14T11:59:27.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:26 smithi093 ceph-mon[128922]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:27.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:26 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/2735021403' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:59:27.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:26 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:27.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:26 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:27.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:26 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:27.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:26 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:59:27.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:26 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:59:27.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:26 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:59:27.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:26 smithi097 ceph-mon[129313]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:27.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:26 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/2735021403' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:59:27.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:26 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:27.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:26 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:27.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:26 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:27.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:26 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:59:27.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:26 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:59:27.387 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:26 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:59:27.394 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:26 smithi017 ceph-mon[126339]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:27.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:26 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/2735021403' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:59:27.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:26 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:27.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:26 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:27.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:26 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:27.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:26 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:59:27.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:26 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:59:27.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:26 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:59:28.227 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:27 smithi097 ceph-mon[129313]: Deploying daemon mds.i on smithi097 2023-07-14T11:59:28.228 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:27 smithi097 ceph-mon[129313]: mds.? [v2:172.21.15.93:6836/1880673911,v1:172.21.15.93:6837/1880673911] up:boot 2023-07-14T11:59:28.228 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:27 smithi097 ceph-mon[129313]: fsmap 8 up:standby 2023-07-14T11:59:28.228 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:27 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-14T11:59:28.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:27 smithi093 ceph-mon[128922]: Deploying daemon mds.i on smithi097 2023-07-14T11:59:28.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:27 smithi093 ceph-mon[128922]: mds.? [v2:172.21.15.93:6836/1880673911,v1:172.21.15.93:6837/1880673911] up:boot 2023-07-14T11:59:28.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:27 smithi093 ceph-mon[128922]: fsmap 8 up:standby 2023-07-14T11:59:28.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:27 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-14T11:59:28.394 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:27 smithi017 ceph-mon[126339]: Deploying daemon mds.i on smithi097 2023-07-14T11:59:28.394 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:27 smithi017 ceph-mon[126339]: mds.? [v2:172.21.15.93:6836/1880673911,v1:172.21.15.93:6837/1880673911] up:boot 2023-07-14T11:59:28.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:27 smithi017 ceph-mon[126339]: fsmap 8 up:standby 2023-07-14T11:59:28.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:27 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-14T11:59:28.447 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:59:28.517 INFO:journalctl@ceph.mds.i.smithi097.stdout:Jul 14 11:59:28 smithi097 systemd[1]: Starting Ceph mds.i for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T11:59:28.662 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.681+0000 7f99247ff700 1 -- 172.21.15.17:0/1520009416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f991c0ffba0 msgr2=0x7f991c0fffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:28.662 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.681+0000 7f99247ff700 1 --2- 172.21.15.17:0/1520009416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f991c0ffba0 0x7f991c0fffb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9918009990 tx=0x7f9918009ca0 comp rx=0 tx=0).stop 2023-07-14T11:59:28.663 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.681+0000 7f99247ff700 1 -- 172.21.15.17:0/1520009416 shutdown_connections 2023-07-14T11:59:28.663 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.681+0000 7f99247ff700 1 --2- 172.21.15.17:0/1520009416 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f991c1017e0 0x7f991c108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:28.663 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.681+0000 7f99247ff700 1 --2- 172.21.15.17:0/1520009416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f991c0ffba0 0x7f991c0fffb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:28.663 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.681+0000 7f99247ff700 1 --2- 172.21.15.17:0/1520009416 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f991c100e30 0x7f991c1012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:28.663 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.681+0000 7f99247ff700 1 -- 172.21.15.17:0/1520009416 >> 172.21.15.17:0/1520009416 conn(0x7f991c0fb1c0 msgr2=0x7f991c0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:28.663 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.681+0000 7f99247ff700 1 -- 172.21.15.17:0/1520009416 shutdown_connections 2023-07-14T11:59:28.664 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.681+0000 7f99247ff700 1 -- 172.21.15.17:0/1520009416 wait complete. 2023-07-14T11:59:28.665 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.682+0000 7f99247ff700 1 Processor -- start 2023-07-14T11:59:28.665 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.683+0000 7f99247ff700 1 -- start start 2023-07-14T11:59:28.666 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.683+0000 7f99247ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f991c0ffba0 0x7f991c197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:28.666 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.683+0000 7f99247ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f991c100e30 0x7f991c198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:28.666 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.683+0000 7f99247ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f991c1017e0 0x7f991c19f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:28.666 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.683+0000 7f99247ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991c19f660 con 0x7f991c100e30 2023-07-14T11:59:28.666 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.683+0000 7f99247ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991c19f7a0 con 0x7f991c0ffba0 2023-07-14T11:59:28.666 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.683+0000 7f99247ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991c19f930 con 0x7f991c1017e0 2023-07-14T11:59:28.666 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.683+0000 7f9921d9a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f991c100e30 0x7f991c198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:28.667 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.683+0000 7f9921d9a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f991c100e30 0x7f991c198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56034/0 (socket says 172.21.15.17:56034) 2023-07-14T11:59:28.667 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.683+0000 7f9921d9a700 1 -- 172.21.15.17:0/4209306476 learned_addr learned my addr 172.21.15.17:0/4209306476 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:59:28.667 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.683+0000 7f9922d9c700 1 --2- 172.21.15.17:0/4209306476 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f991c1017e0 0x7f991c19f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:28.667 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.683+0000 7f992259b700 1 --2- 172.21.15.17:0/4209306476 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f991c0ffba0 0x7f991c197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:28.667 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.684+0000 7f9921d9a700 1 -- 172.21.15.17:0/4209306476 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f991c1017e0 msgr2=0x7f991c19f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:28.667 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.684+0000 7f9921d9a700 1 --2- 172.21.15.17:0/4209306476 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f991c1017e0 0x7f991c19f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:28.667 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.684+0000 7f9921d9a700 1 -- 172.21.15.17:0/4209306476 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f991c0ffba0 msgr2=0x7f991c197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:28.667 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.684+0000 7f9921d9a700 1 --2- 172.21.15.17:0/4209306476 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f991c0ffba0 0x7f991c197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:28.668 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.684+0000 7f9921d9a700 1 -- 172.21.15.17:0/4209306476 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99180095f0 con 0x7f991c100e30 2023-07-14T11:59:28.668 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.684+0000 7f9922d9c700 1 --2- 172.21.15.17:0/4209306476 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f991c1017e0 0x7f991c19f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:59:28.668 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.684+0000 7f9921d9a700 1 --2- 172.21.15.17:0/4209306476 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f991c100e30 0x7f991c198020 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f990c00d980 tx=0x7f990c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:28.668 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.684+0000 7f99137fe700 1 -- 172.21.15.17:0/4209306476 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c004220 con 0x7f991c100e30 2023-07-14T11:59:28.668 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.684+0000 7f99247ff700 1 -- 172.21.15.17:0/4209306476 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f991c19fc10 con 0x7f991c100e30 2023-07-14T11:59:28.669 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.684+0000 7f99137fe700 1 -- 172.21.15.17:0/4209306476 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f990c007640 con 0x7f991c100e30 2023-07-14T11:59:28.669 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.684+0000 7f99137fe700 1 -- 172.21.15.17:0/4209306476 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c005690 con 0x7f991c100e30 2023-07-14T11:59:28.669 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.684+0000 7f99247ff700 1 -- 172.21.15.17:0/4209306476 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f991c1a0170 con 0x7f991c100e30 2023-07-14T11:59:28.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.685+0000 7f99247ff700 1 -- 172.21.15.17:0/4209306476 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f991c192280 con 0x7f991c100e30 2023-07-14T11:59:28.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.687+0000 7f99137fe700 1 -- 172.21.15.17:0/4209306476 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f990c004750 con 0x7f991c100e30 2023-07-14T11:59:28.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.689+0000 7f99137fe700 1 --2- 172.21.15.17:0/4209306476 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f99080a1150 0x7f99080a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:28.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.689+0000 7f99137fe700 1 -- 172.21.15.17:0/4209306476 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f990c0cc330 con 0x7f991c100e30 2023-07-14T11:59:28.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.689+0000 7f992259b700 1 --2- 172.21.15.17:0/4209306476 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f99080a1150 0x7f99080a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:28.671 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.690+0000 7f992259b700 1 --2- 172.21.15.17:0/4209306476 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f99080a1150 0x7f99080a3600 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f991800b2e0 tx=0x7f9918005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:28.674 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.693+0000 7f99137fe700 1 -- 172.21.15.17:0/4209306476 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f990c095e80 con 0x7f991c100e30 2023-07-14T11:59:28.781 INFO:journalctl@ceph.mds.i.smithi097.stdout:Jul 14 11:59:28 smithi097 podman[155664]: 2023-07-14T11:59:28.928 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.948+0000 7f99247ff700 1 -- 172.21.15.17:0/4209306476 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f991c02c850 con 0x7f991c100e30 2023-07-14T11:59:28.929 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.948+0000 7f99137fe700 1 -- 172.21.15.17:0/4209306476 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6541 (secure 0 0 0) 0x7f990c0b9030 con 0x7f991c100e30 2023-07-14T11:59:28.930 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:59:28.930 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/55384193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":55384193},{"type":"v1","addr":"172.21.15.17:6835","nonce":55384193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/424956554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":424956554},{"type":"v1","addr":"172.21.15.17:6837","nonce":424956554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/326319712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":326319712},{"type":"v1","addr":"172.21.15.93:6833","nonce":326319712}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/878913679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":878913679},{"type":"v1","addr":"172.21.15.97:6833","nonce":878913679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/2779420503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":2779420503},{"type":"v1","addr":"172.21.15.93:6835","nonce":2779420503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2740799393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2740799393},{"type":"v1","addr":"172.21.15.97:6835","nonce":2740799393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24509,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/1880673911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":1880673911},{"type":"v1","addr":"172.21.15.93:6837","nonce":1880673911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3684187131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3684187131},{"type":"v1","addr":"172.21.15.97:6837","nonce":3684187131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2023-07-14T11:59:28.931 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.951+0000 7f99247ff700 1 -- 172.21.15.17:0/4209306476 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f99080a1150 msgr2=0x7f99080a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:28.932 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.951+0000 7f99247ff700 1 --2- 172.21.15.17:0/4209306476 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f99080a1150 0x7f99080a3600 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f991800b2e0 tx=0x7f9918005fb0 comp rx=0 tx=0).stop 2023-07-14T11:59:28.932 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.951+0000 7f99247ff700 1 -- 172.21.15.17:0/4209306476 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f991c100e30 msgr2=0x7f991c198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:28.932 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.951+0000 7f99247ff700 1 --2- 172.21.15.17:0/4209306476 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f991c100e30 0x7f991c198020 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f990c00d980 tx=0x7f990c00dc90 comp rx=0 tx=0).stop 2023-07-14T11:59:28.932 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.951+0000 7f99247ff700 1 -- 172.21.15.17:0/4209306476 shutdown_connections 2023-07-14T11:59:28.932 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.951+0000 7f99247ff700 1 --2- 172.21.15.17:0/4209306476 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f99080a1150 0x7f99080a3600 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:28.933 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.951+0000 7f99247ff700 1 --2- 172.21.15.17:0/4209306476 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f991c0ffba0 0x7f991c197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:28.933 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.951+0000 7f99247ff700 1 --2- 172.21.15.17:0/4209306476 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f991c1017e0 0x7f991c19f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:28.933 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.951+0000 7f99247ff700 1 --2- 172.21.15.17:0/4209306476 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f991c100e30 0x7f991c198020 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:28.933 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.951+0000 7f99247ff700 1 -- 172.21.15.17:0/4209306476 >> 172.21.15.17:0/4209306476 conn(0x7f991c0fb1c0 msgr2=0x7f991c1066f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:28.933 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.951+0000 7f99247ff700 1 -- 172.21.15.17:0/4209306476 shutdown_connections 2023-07-14T11:59:28.933 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:28.951+0000 7f99247ff700 1 -- 172.21.15.17:0/4209306476 wait complete. 2023-07-14T11:59:28.934 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 9 2023-07-14T11:59:29.134 INFO:journalctl@ceph.mds.i.smithi097.stdout:Jul 14 11:59:28 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-i[155680]: starting mds.i at 2023-07-14T11:59:29.135 INFO:journalctl@ceph.mds.i.smithi097.stdout:Jul 14 11:59:28 smithi097 bash[155664]: 31e2f76bdba4f96295834fd3c376c4177e48436a30711d59de6c9a58a259b026 2023-07-14T11:59:29.135 INFO:journalctl@ceph.mds.i.smithi097.stdout:Jul 14 11:59:28 smithi097 systemd[1]: Started Ceph mds.i for b3c386cc-223c-11ee-9b33-001a4aab830c. 2023-07-14T11:59:29.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:28 smithi097 ceph-mon[129313]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:29.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:28 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:29.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:28 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:29.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:28 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:29.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:28 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:59:29.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:28 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:59:29.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:28 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:59:29.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:28 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/4209306476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:59:29.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:28 smithi093 ceph-mon[128922]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:29.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:28 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:29.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:28 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:29.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:28 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:29.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:28 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:59:29.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:28 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:59:29.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:28 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:59:29.363 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:28 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/4209306476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:59:29.394 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:28 smithi017 ceph-mon[126339]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:29.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:28 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:29.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:28 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:29.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:28 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:29.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:28 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:59:29.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:28 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:59:29.395 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:28 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:59:29.396 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:28 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/4209306476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:59:30.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:29 smithi017 ceph-mon[126339]: Deploying daemon mds.g on smithi017 2023-07-14T11:59:30.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:29 smithi017 ceph-mon[126339]: mds.? [v2:172.21.15.97:6838/3267815331,v1:172.21.15.97:6839/3267815331] up:boot 2023-07-14T11:59:30.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:29 smithi017 ceph-mon[126339]: fsmap 9 up:standby 2023-07-14T11:59:30.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:29 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-14T11:59:30.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:29 smithi093 ceph-mon[128922]: Deploying daemon mds.g on smithi017 2023-07-14T11:59:30.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:29 smithi093 ceph-mon[128922]: mds.? [v2:172.21.15.97:6838/3267815331,v1:172.21.15.97:6839/3267815331] up:boot 2023-07-14T11:59:30.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:29 smithi093 ceph-mon[128922]: fsmap 9 up:standby 2023-07-14T11:59:30.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:29 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-14T11:59:30.384 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:29 smithi097 ceph-mon[129313]: Deploying daemon mds.g on smithi017 2023-07-14T11:59:30.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:29 smithi097 ceph-mon[129313]: mds.? [v2:172.21.15.97:6838/3267815331,v1:172.21.15.97:6839/3267815331] up:boot 2023-07-14T11:59:30.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:29 smithi097 ceph-mon[129313]: fsmap 9 up:standby 2023-07-14T11:59:30.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:29 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-14T11:59:30.951 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:59:31.121 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:30 smithi017 ceph-mon[126339]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:31.278 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.297+0000 7f27d0bff700 1 -- 172.21.15.17:0/126045247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27cc1017e0 msgr2=0x7f27cc108080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:31.278 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.297+0000 7f27d0bff700 1 --2- 172.21.15.17:0/126045247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27cc1017e0 0x7f27cc108080 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f27c4009f90 tx=0x7f27c400b330 comp rx=0 tx=0).stop 2023-07-14T11:59:31.278 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.297+0000 7f27d0bff700 1 -- 172.21.15.17:0/126045247 shutdown_connections 2023-07-14T11:59:31.278 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.297+0000 7f27d0bff700 1 --2- 172.21.15.17:0/126045247 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27cc0ffba0 0x7f27cc0fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:31.279 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.297+0000 7f27d0bff700 1 --2- 172.21.15.17:0/126045247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27cc1017e0 0x7f27cc108080 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:31.279 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.297+0000 7f27d0bff700 1 --2- 172.21.15.17:0/126045247 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27cc100e30 0x7f27cc1012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:31.279 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.297+0000 7f27d0bff700 1 -- 172.21.15.17:0/126045247 >> 172.21.15.17:0/126045247 conn(0x7f27cc0fb1c0 msgr2=0x7f27cc0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:31.279 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.297+0000 7f27d0bff700 1 -- 172.21.15.17:0/126045247 shutdown_connections 2023-07-14T11:59:31.279 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.297+0000 7f27d0bff700 1 -- 172.21.15.17:0/126045247 wait complete. 2023-07-14T11:59:31.282 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.298+0000 7f27d0bff700 1 Processor -- start 2023-07-14T11:59:31.282 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.298+0000 7f27d0bff700 1 -- start start 2023-07-14T11:59:31.282 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.298+0000 7f27d0bff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27cc0ffba0 0x7f27cc1059c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:31.283 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.298+0000 7f27d0bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27cc100e30 0x7f27cc105f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:31.283 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.299+0000 7f27d0bff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27cc1017e0 0x7f27cc102550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:31.283 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.299+0000 7f27d0bff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc102cf0 con 0x7f27cc0ffba0 2023-07-14T11:59:31.283 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.299+0000 7f27d0bff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc102e60 con 0x7f27cc1017e0 2023-07-14T11:59:31.284 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.299+0000 7f27d0bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc103020 con 0x7f27cc100e30 2023-07-14T11:59:31.284 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.299+0000 7f27ca59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27cc0ffba0 0x7f27cc1059c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:31.284 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.299+0000 7f27c9d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27cc100e30 0x7f27cc105f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:31.285 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.299+0000 7f27ca59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27cc0ffba0 0x7f27cc1059c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56052/0 (socket says 172.21.15.17:56052) 2023-07-14T11:59:31.285 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.299+0000 7f27ca59c700 1 -- 172.21.15.17:0/1535412643 learned_addr learned my addr 172.21.15.17:0/1535412643 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:59:31.285 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.299+0000 7f27cad9d700 1 --2- 172.21.15.17:0/1535412643 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27cc1017e0 0x7f27cc102550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:31.285 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.299+0000 7f27c9d9b700 1 -- 172.21.15.17:0/1535412643 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27cc1017e0 msgr2=0x7f27cc102550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:31.286 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.299+0000 7f27c9d9b700 1 --2- 172.21.15.17:0/1535412643 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27cc1017e0 0x7f27cc102550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:31.286 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.299+0000 7f27c9d9b700 1 -- 172.21.15.17:0/1535412643 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27cc0ffba0 msgr2=0x7f27cc1059c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:31.286 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.299+0000 7f27c9d9b700 1 --2- 172.21.15.17:0/1535412643 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27cc0ffba0 0x7f27cc1059c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:31.287 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.299+0000 7f27c9d9b700 1 -- 172.21.15.17:0/1535412643 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c4009c40 con 0x7f27cc100e30 2023-07-14T11:59:31.287 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.300+0000 7f27c9d9b700 1 --2- 172.21.15.17:0/1535412643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27cc100e30 0x7f27cc105f00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f27c000e9c0 tx=0x7f27c000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:31.287 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.300+0000 7f27bb7fe700 1 -- 172.21.15.17:0/1535412643 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c000c9c0 con 0x7f27cc100e30 2023-07-14T11:59:31.288 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.300+0000 7f27bb7fe700 1 -- 172.21.15.17:0/1535412643 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27c0010430 con 0x7f27cc100e30 2023-07-14T11:59:31.288 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.300+0000 7f27bb7fe700 1 -- 172.21.15.17:0/1535412643 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c0018680 con 0x7f27cc100e30 2023-07-14T11:59:31.288 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.300+0000 7f27d0bff700 1 -- 172.21.15.17:0/1535412643 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27cc1a4160 con 0x7f27cc100e30 2023-07-14T11:59:31.289 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.300+0000 7f27d0bff700 1 -- 172.21.15.17:0/1535412643 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27cc1a45f0 con 0x7f27cc100e30 2023-07-14T11:59:31.290 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.301+0000 7f27d0bff700 1 -- 172.21.15.17:0/1535412643 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27cc10b3c0 con 0x7f27cc100e30 2023-07-14T11:59:31.290 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.303+0000 7f27bb7fe700 1 -- 172.21.15.17:0/1535412643 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f27c0010860 con 0x7f27cc100e30 2023-07-14T11:59:31.291 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.305+0000 7f27bb7fe700 1 --2- 172.21.15.17:0/1535412643 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f27b40a11d0 0x7f27b40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:31.291 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.305+0000 7f27bb7fe700 1 -- 172.21.15.17:0/1535412643 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f27c0014070 con 0x7f27cc100e30 2023-07-14T11:59:31.291 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.305+0000 7f27ca59c700 1 --2- 172.21.15.17:0/1535412643 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f27b40a11d0 0x7f27b40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:31.291 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.305+0000 7f27ca59c700 1 --2- 172.21.15.17:0/1535412643 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f27b40a11d0 0x7f27b40a3680 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f27bc009d20 tx=0x7f27bc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:31.292 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.308+0000 7f27bb7fe700 1 -- 172.21.15.17:0/1535412643 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f27c0051790 con 0x7f27cc100e30 2023-07-14T11:59:31.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:30 smithi093 ceph-mon[128922]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:31.384 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:30 smithi097 ceph-mon[129313]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:31.394 INFO:journalctl@ceph.mds.g.smithi017.stdout:Jul 14 11:59:31 smithi017 systemd[1]: Starting Ceph mds.g for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T11:59:31.585 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.604+0000 7f27d0bff700 1 -- 172.21.15.17:0/1535412643 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f27cc1a4c30 con 0x7f27cc100e30 2023-07-14T11:59:31.586 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.605+0000 7f27bb7fe700 1 -- 172.21.15.17:0/1535412643 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7300 (secure 0 0 0) 0x7f27c0096000 con 0x7f27cc100e30 2023-07-14T11:59:31.586 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:59:31.587 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/55384193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":55384193},{"type":"v1","addr":"172.21.15.17:6835","nonce":55384193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/424956554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":424956554},{"type":"v1","addr":"172.21.15.17:6837","nonce":424956554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/326319712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":326319712},{"type":"v1","addr":"172.21.15.93:6833","nonce":326319712}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/878913679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":878913679},{"type":"v1","addr":"172.21.15.97:6833","nonce":878913679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/2779420503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":2779420503},{"type":"v1","addr":"172.21.15.93:6835","nonce":2779420503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2740799393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2740799393},{"type":"v1","addr":"172.21.15.97:6835","nonce":2740799393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24509,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/1880673911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":1880673911},{"type":"v1","addr":"172.21.15.93:6837","nonce":1880673911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3684187131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3684187131},{"type":"v1","addr":"172.21.15.97:6837","nonce":3684187131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24541,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/3267815331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3267815331},{"type":"v1","addr":"172.21.15.97:6839","nonce":3267815331}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-07-14T11:59:31.588 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.608+0000 7f27d0bff700 1 -- 172.21.15.17:0/1535412643 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f27b40a11d0 msgr2=0x7f27b40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:31.589 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.608+0000 7f27d0bff700 1 --2- 172.21.15.17:0/1535412643 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f27b40a11d0 0x7f27b40a3680 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f27bc009d20 tx=0x7f27bc009260 comp rx=0 tx=0).stop 2023-07-14T11:59:31.589 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.608+0000 7f27d0bff700 1 -- 172.21.15.17:0/1535412643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27cc100e30 msgr2=0x7f27cc105f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:31.589 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.608+0000 7f27d0bff700 1 --2- 172.21.15.17:0/1535412643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27cc100e30 0x7f27cc105f00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f27c000e9c0 tx=0x7f27c000ecd0 comp rx=0 tx=0).stop 2023-07-14T11:59:31.589 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.608+0000 7f27d0bff700 1 -- 172.21.15.17:0/1535412643 shutdown_connections 2023-07-14T11:59:31.589 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.608+0000 7f27d0bff700 1 --2- 172.21.15.17:0/1535412643 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f27b40a11d0 0x7f27b40a3680 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:31.590 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.608+0000 7f27d0bff700 1 --2- 172.21.15.17:0/1535412643 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27cc1017e0 0x7f27cc102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:31.590 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.608+0000 7f27d0bff700 1 --2- 172.21.15.17:0/1535412643 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27cc0ffba0 0x7f27cc1059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:31.590 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.608+0000 7f27d0bff700 1 --2- 172.21.15.17:0/1535412643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27cc100e30 0x7f27cc105f00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:31.590 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.608+0000 7f27d0bff700 1 -- 172.21.15.17:0/1535412643 >> 172.21.15.17:0/1535412643 conn(0x7f27cc0fb1c0 msgr2=0x7f27cc0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:31.590 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.608+0000 7f27d0bff700 1 -- 172.21.15.17:0/1535412643 shutdown_connections 2023-07-14T11:59:31.590 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:31.608+0000 7f27d0bff700 1 -- 172.21.15.17:0/1535412643 wait complete. 2023-07-14T11:59:31.591 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 10 2023-07-14T11:59:31.894 INFO:journalctl@ceph.mds.g.smithi017.stdout:Jul 14 11:59:31 smithi017 podman[160778]: 2023-07-14T11:59:32.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:31 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/1535412643' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:59:32.384 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:31 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/1535412643' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:59:32.394 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:31 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/1535412643' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:59:32.394 INFO:journalctl@ceph.mds.g.smithi017.stdout:Jul 14 11:59:31 smithi017 bash[160778]: 77724550bd9bc69d701fffda52cd9dfd33d92b23e6418fdc8e97f14ecd42ee39 2023-07-14T11:59:32.394 INFO:journalctl@ceph.mds.g.smithi017.stdout:Jul 14 11:59:31 smithi017 ceph-mds[160820]: -- 172.21.15.17:0/1630690957 <== mon.0 v2:172.21.15.17:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563606b371e0 con 0x563606b0c800 2023-07-14T11:59:32.395 INFO:journalctl@ceph.mds.g.smithi017.stdout:Jul 14 11:59:31 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-g[160796]: starting mds.g at 2023-07-14T11:59:32.395 INFO:journalctl@ceph.mds.g.smithi017.stdout:Jul 14 11:59:32 smithi017 systemd[1]: Started Ceph mds.g for b3c386cc-223c-11ee-9b33-001a4aab830c. 2023-07-14T11:59:33.262 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:32 smithi017 ceph-mon[126339]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:33.262 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:32 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:33.262 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:32 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:33.262 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:32 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:33.262 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:32 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:59:33.263 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:32 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:59:33.263 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:32 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:59:33.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:32 smithi093 ceph-mon[128922]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:33.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:32 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:33.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:32 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:33.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:32 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:33.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:32 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:59:33.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:32 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:59:33.362 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:32 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:59:33.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:32 smithi097 ceph-mon[129313]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:33.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:32 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:33.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:32 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:33.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:32 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:33.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:32 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:59:33.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:32 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:59:33.386 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:32 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:59:33.608 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:59:33.823 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:33.841+0000 7f72d89ff700 1 -- 172.21.15.17:0/2338109602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72d41017f0 msgr2=0x7f72d4108090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:33.823 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:33.841+0000 7f72d89ff700 1 --2- 172.21.15.17:0/2338109602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72d41017f0 0x7f72d4108090 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f72cc00b0a0 tx=0x7f72cc00b3b0 comp rx=0 tx=0).stop 2023-07-14T11:59:33.823 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:33.842+0000 7f72d89ff700 1 -- 172.21.15.17:0/2338109602 shutdown_connections 2023-07-14T11:59:33.823 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:33.842+0000 7f72d89ff700 1 --2- 172.21.15.17:0/2338109602 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f72d40ffbb0 0x7f72d40fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:33.823 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:33.842+0000 7f72d89ff700 1 --2- 172.21.15.17:0/2338109602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72d41017f0 0x7f72d4108090 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:33.824 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:33.842+0000 7f72d89ff700 1 --2- 172.21.15.17:0/2338109602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72d4100e40 0x7f72d41012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:33.824 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:33.842+0000 7f72d89ff700 1 -- 172.21.15.17:0/2338109602 >> 172.21.15.17:0/2338109602 conn(0x7f72d40fb1d0 msgr2=0x7f72d40fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:33.824 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:33.842+0000 7f72d89ff700 1 -- 172.21.15.17:0/2338109602 shutdown_connections 2023-07-14T11:59:33.824 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:33.842+0000 7f72d89ff700 1 -- 172.21.15.17:0/2338109602 wait complete. 2023-07-14T11:59:33.826 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:33.843+0000 7f72d89ff700 1 Processor -- start 2023-07-14T11:59:33.826 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:33.843+0000 7f72d89ff700 1 -- start start 2023-07-14T11:59:33.826 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:33.843+0000 7f72d89ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f72d40ffbb0 0x7f72d41959e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:33.826 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:33.843+0000 7f72d89ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72d4100e40 0x7f72d4195f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:33.826 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:33.843+0000 7f72d89ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72d41017f0 0x7f72d419cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:33.826 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:33.843+0000 7f72d89ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d419d560 con 0x7f72d4100e40 2023-07-14T11:59:33.827 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:33.843+0000 7f72d89ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d419d6a0 con 0x7f72d40ffbb0 2023-07-14T11:59:33.827 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:33.843+0000 7f72d89ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d419d830 con 0x7f72d41017f0 2023-07-14T11:59:33.827 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:33.844+0000 7f72d259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f72d40ffbb0 0x7f72d41959e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:33.827 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:33.844+0000 7f72d259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f72d40ffbb0 0x7f72d41959e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.17:40432/0 (socket says 172.21.15.17:40432) 2023-07-14T11:59:33.828 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:33.844+0000 7f72d259c700 1 -- 172.21.15.17:0/3367332531 learned_addr learned my addr 172.21.15.17:0/3367332531 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:59:33.828 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:33.844+0000 7f72cbfff700 1 --2- 172.21.15.17:0/3367332531 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72d4100e40 0x7f72d4195f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:33.828 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:33.844+0000 7f72d2d9d700 1 --2- 172.21.15.17:0/3367332531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72d41017f0 0x7f72d419cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:33.828 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:33.844+0000 7f72d259c700 1 -- 172.21.15.17:0/3367332531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72d41017f0 msgr2=0x7f72d419cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:33.828 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:33.844+0000 7f72d259c700 1 --2- 172.21.15.17:0/3367332531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72d41017f0 0x7f72d419cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:33.828 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:33.844+0000 7f72d259c700 1 -- 172.21.15.17:0/3367332531 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72d4100e40 msgr2=0x7f72d4195f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:33.829 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:33.844+0000 7f72d259c700 1 --2- 172.21.15.17:0/3367332531 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72d4100e40 0x7f72d4195f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:33.829 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:33.844+0000 7f72d259c700 1 -- 172.21.15.17:0/3367332531 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72cc009d00 con 0x7f72d40ffbb0 2023-07-14T11:59:33.829 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:33.844+0000 7f72d2d9d700 1 --2- 172.21.15.17:0/3367332531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72d41017f0 0x7f72d419cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:59:33.829 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:33.844+0000 7f72cbfff700 1 --2- 172.21.15.17:0/3367332531 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72d4100e40 0x7f72d4195f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:59:33.829 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:33.844+0000 7f72d259c700 1 --2- 172.21.15.17:0/3367332531 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f72d40ffbb0 0x7f72d41959e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f72c400e9a0 tx=0x7f72c400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:33.829 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:33.844+0000 7f72cb7fe700 1 -- 172.21.15.17:0/3367332531 <== mon.2 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c4009790 con 0x7f72d40ffbb0 2023-07-14T11:59:33.829 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:33.845+0000 7f72d89ff700 1 -- 172.21.15.17:0/3367332531 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72d419db10 con 0x7f72d40ffbb0 2023-07-14T11:59:33.830 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:33.845+0000 7f72cb7fe700 1 -- 172.21.15.17:0/3367332531 <== mon.2 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72c400cbd0 con 0x7f72d40ffbb0 2023-07-14T11:59:33.830 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:33.845+0000 7f72cb7fe700 1 -- 172.21.15.17:0/3367332531 <== mon.2 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c4018680 con 0x7f72d40ffbb0 2023-07-14T11:59:33.830 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:33.845+0000 7f72d89ff700 1 -- 172.21.15.17:0/3367332531 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72d419e070 con 0x7f72d40ffbb0 2023-07-14T11:59:33.831 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:33.846+0000 7f72d89ff700 1 -- 172.21.15.17:0/3367332531 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72d4106ab0 con 0x7f72d40ffbb0 2023-07-14T11:59:33.831 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:33.848+0000 7f72cb7fe700 1 -- 172.21.15.17:0/3367332531 <== mon.2 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f72c400cd40 con 0x7f72d40ffbb0 2023-07-14T11:59:33.831 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:33.849+0000 7f72cb7fe700 1 --2- 172.21.15.17:0/3367332531 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f72c00a1150 0x7f72c00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:33.831 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:33.849+0000 7f72cb7fe700 1 -- 172.21.15.17:0/3367332531 <== mon.2 v2:172.21.15.93:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f72c4014070 con 0x7f72d40ffbb0 2023-07-14T11:59:33.831 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:33.850+0000 7f72cbfff700 1 --2- 172.21.15.17:0/3367332531 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f72c00a1150 0x7f72c00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:33.831 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:33.850+0000 7f72cbfff700 1 --2- 172.21.15.17:0/3367332531 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f72c00a1150 0x7f72c00a3600 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f72bc011490 tx=0x7f72bc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:33.832 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:33.851+0000 7f72cb7fe700 1 -- 172.21.15.17:0/3367332531 <== mon.2 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f72c40963b0 con 0x7f72d40ffbb0 2023-07-14T11:59:34.104 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:34.123+0000 7f72d89ff700 1 -- 172.21.15.17:0/3367332531 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72d402ccc0 con 0x7f72d40ffbb0 2023-07-14T11:59:34.105 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:34.124+0000 7f72cb7fe700 1 -- 172.21.15.17:0/3367332531 <== mon.2 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8058 (secure 0 0 0) 0x7f72c40999d0 con 0x7f72d40ffbb0 2023-07-14T11:59:34.109 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:59:34.109 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/55384193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":55384193},{"type":"v1","addr":"172.21.15.17:6835","nonce":55384193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/424956554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":424956554},{"type":"v1","addr":"172.21.15.17:6837","nonce":424956554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/326319712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":326319712},{"type":"v1","addr":"172.21.15.93:6833","nonce":326319712}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/878913679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":878913679},{"type":"v1","addr":"172.21.15.97:6833","nonce":878913679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/2779420503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":2779420503},{"type":"v1","addr":"172.21.15.93:6835","nonce":2779420503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2740799393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2740799393},{"type":"v1","addr":"172.21.15.97:6835","nonce":2740799393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24509,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/1880673911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":1880673911},{"type":"v1","addr":"172.21.15.93:6837","nonce":1880673911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3684187131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3684187131},{"type":"v1","addr":"172.21.15.97:6837","nonce":3684187131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24533,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6839/2976493015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":2976493015},{"type":"v1","addr":"172.21.15.17:6839","nonce":2976493015}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24541,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/3267815331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3267815331},{"type":"v1","addr":"172.21.15.97:6839","nonce":3267815331}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-07-14T11:59:34.110 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:34.126+0000 7f72d89ff700 1 -- 172.21.15.17:0/3367332531 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f72c00a1150 msgr2=0x7f72c00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:34.110 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:34.126+0000 7f72d89ff700 1 --2- 172.21.15.17:0/3367332531 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f72c00a1150 0x7f72c00a3600 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f72bc011490 tx=0x7f72bc009260 comp rx=0 tx=0).stop 2023-07-14T11:59:34.110 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:34.126+0000 7f72d89ff700 1 -- 172.21.15.17:0/3367332531 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f72d40ffbb0 msgr2=0x7f72d41959e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:34.111 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:34.126+0000 7f72d89ff700 1 --2- 172.21.15.17:0/3367332531 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f72d40ffbb0 0x7f72d41959e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f72c400e9a0 tx=0x7f72c400ecb0 comp rx=0 tx=0).stop 2023-07-14T11:59:34.111 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:34.127+0000 7f72d89ff700 1 -- 172.21.15.17:0/3367332531 shutdown_connections 2023-07-14T11:59:34.111 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:34.127+0000 7f72d89ff700 1 --2- 172.21.15.17:0/3367332531 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f72c00a1150 0x7f72c00a3600 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:34.111 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:34.127+0000 7f72d89ff700 1 --2- 172.21.15.17:0/3367332531 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f72d40ffbb0 0x7f72d41959e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:34.111 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:34.127+0000 7f72d89ff700 1 --2- 172.21.15.17:0/3367332531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72d41017f0 0x7f72d419cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:34.111 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:34.127+0000 7f72d89ff700 1 --2- 172.21.15.17:0/3367332531 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72d4100e40 0x7f72d4195f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:34.111 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:34.127+0000 7f72d89ff700 1 -- 172.21.15.17:0/3367332531 >> 172.21.15.17:0/3367332531 conn(0x7f72d40fb1d0 msgr2=0x7f72d40fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:34.112 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:34.127+0000 7f72d89ff700 1 -- 172.21.15.17:0/3367332531 shutdown_connections 2023-07-14T11:59:34.112 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:34.127+0000 7f72d89ff700 1 -- 172.21.15.17:0/3367332531 wait complete. 2023-07-14T11:59:34.112 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 11 2023-07-14T11:59:34.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:33 smithi093 ceph-mon[128922]: Deploying daemon mds.a on smithi017 2023-07-14T11:59:34.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:33 smithi093 ceph-mon[128922]: mds.? [v2:172.21.15.17:6838/2976493015,v1:172.21.15.17:6839/2976493015] up:boot 2023-07-14T11:59:34.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:33 smithi093 ceph-mon[128922]: fsmap 10 up:standby 2023-07-14T11:59:34.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:33 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-14T11:59:34.384 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:33 smithi097 ceph-mon[129313]: Deploying daemon mds.a on smithi017 2023-07-14T11:59:34.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:33 smithi097 ceph-mon[129313]: mds.? [v2:172.21.15.17:6838/2976493015,v1:172.21.15.17:6839/2976493015] up:boot 2023-07-14T11:59:34.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:33 smithi097 ceph-mon[129313]: fsmap 10 up:standby 2023-07-14T11:59:34.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:33 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-14T11:59:34.394 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:33 smithi017 ceph-mon[126339]: Deploying daemon mds.a on smithi017 2023-07-14T11:59:34.394 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:33 smithi017 ceph-mon[126339]: mds.? [v2:172.21.15.17:6838/2976493015,v1:172.21.15.17:6839/2976493015] up:boot 2023-07-14T11:59:34.394 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:33 smithi017 ceph-mon[126339]: fsmap 10 up:standby 2023-07-14T11:59:34.394 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:33 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-14T11:59:34.894 INFO:journalctl@ceph.mds.a.smithi017.stdout:Jul 14 11:59:34 smithi017 systemd[1]: Starting Ceph mds.a for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T11:59:35.161 INFO:journalctl@ceph.mds.a.smithi017.stdout:Jul 14 11:59:34 smithi017 podman[161167]: 2023-07-14T11:59:35.161 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:34 smithi017 ceph-mon[126339]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:35.161 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:34 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/3367332531' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:59:35.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:34 smithi093 ceph-mon[128922]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:35.361 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:34 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/3367332531' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:59:35.384 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:34 smithi097 ceph-mon[129313]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:35.385 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:34 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/3367332531' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:59:35.644 INFO:journalctl@ceph.mds.a.smithi017.stdout:Jul 14 11:59:35 smithi017 ceph-mds[161207]: -- 172.21.15.17:0/4128096264 <== mon.0 v2:172.21.15.17:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564213f8f1e0 con 0x564213f65000 2023-07-14T11:59:35.645 INFO:journalctl@ceph.mds.a.smithi017.stdout:Jul 14 11:59:35 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-a[161183]: starting mds.a at 2023-07-14T11:59:35.645 INFO:journalctl@ceph.mds.a.smithi017.stdout:Jul 14 11:59:35 smithi017 bash[161167]: 815e6d261378dfaf698a2f8223cd9ff362d434a69a59720f27155f3e7ae2eaa2 2023-07-14T11:59:35.645 INFO:journalctl@ceph.mds.a.smithi017.stdout:Jul 14 11:59:35 smithi017 systemd[1]: Started Ceph mds.a for b3c386cc-223c-11ee-9b33-001a4aab830c. 2023-07-14T11:59:36.128 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:59:36.350 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.369+0000 7f4d503ff700 1 -- 172.21.15.17:0/3830310555 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4d480ffba0 msgr2=0x7f4d480fffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:36.350 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.369+0000 7f4d503ff700 1 --2- 172.21.15.17:0/3830310555 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4d480ffba0 0x7f4d480fffb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4d44009990 tx=0x7f4d44009ca0 comp rx=0 tx=0).stop 2023-07-14T11:59:36.351 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.369+0000 7f4d503ff700 1 -- 172.21.15.17:0/3830310555 shutdown_connections 2023-07-14T11:59:36.351 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.369+0000 7f4d503ff700 1 --2- 172.21.15.17:0/3830310555 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4d480ffba0 0x7f4d480fffb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:36.351 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.369+0000 7f4d503ff700 1 --2- 172.21.15.17:0/3830310555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d481017e0 0x7f4d48108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:36.351 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.369+0000 7f4d503ff700 1 --2- 172.21.15.17:0/3830310555 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d48100e30 0x7f4d481012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:36.351 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.369+0000 7f4d503ff700 1 -- 172.21.15.17:0/3830310555 >> 172.21.15.17:0/3830310555 conn(0x7f4d480fb1c0 msgr2=0x7f4d480fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:36.351 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.369+0000 7f4d503ff700 1 -- 172.21.15.17:0/3830310555 shutdown_connections 2023-07-14T11:59:36.351 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.369+0000 7f4d503ff700 1 -- 172.21.15.17:0/3830310555 wait complete. 2023-07-14T11:59:36.353 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.370+0000 7f4d503ff700 1 Processor -- start 2023-07-14T11:59:36.353 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.370+0000 7f4d503ff700 1 -- start start 2023-07-14T11:59:36.353 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.371+0000 7f4d503ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d480ffba0 0x7f4d480742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:36.353 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.371+0000 7f4d503ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d48100e30 0x7f4d48072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:36.354 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.371+0000 7f4d503ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4d481017e0 0x7f4d48072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:36.354 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.371+0000 7f4d503ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d480735e0 con 0x7f4d48100e30 2023-07-14T11:59:36.354 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.371+0000 7f4d503ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d48073750 con 0x7f4d481017e0 2023-07-14T11:59:36.354 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.371+0000 7f4d503ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4819f950 con 0x7f4d480ffba0 2023-07-14T11:59:36.354 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.371+0000 7f4d4d99a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d48100e30 0x7f4d48072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:36.354 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.371+0000 7f4d4e99c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4d481017e0 0x7f4d48072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:36.354 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.371+0000 7f4d4d99a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d48100e30 0x7f4d48072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51750/0 (socket says 172.21.15.17:51750) 2023-07-14T11:59:36.355 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.371+0000 7f4d4d99a700 1 -- 172.21.15.17:0/1516452022 learned_addr learned my addr 172.21.15.17:0/1516452022 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:59:36.355 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.371+0000 7f4d4e19b700 1 --2- 172.21.15.17:0/1516452022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d480ffba0 0x7f4d480742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:36.355 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.371+0000 7f4d4e99c700 1 -- 172.21.15.17:0/1516452022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d480ffba0 msgr2=0x7f4d480742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:36.355 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.371+0000 7f4d4e99c700 1 --2- 172.21.15.17:0/1516452022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d480ffba0 0x7f4d480742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:36.355 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.371+0000 7f4d4e99c700 1 -- 172.21.15.17:0/1516452022 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d48100e30 msgr2=0x7f4d48072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:36.355 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.371+0000 7f4d4e99c700 1 --2- 172.21.15.17:0/1516452022 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d48100e30 0x7f4d48072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:36.355 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.371+0000 7f4d4e99c700 1 -- 172.21.15.17:0/1516452022 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d440095f0 con 0x7f4d481017e0 2023-07-14T11:59:36.355 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.372+0000 7f4d4e99c700 1 --2- 172.21.15.17:0/1516452022 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4d481017e0 0x7f4d48072e40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4d4000efc0 tx=0x7f4d4000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:36.356 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.372+0000 7f4d3f7fe700 1 -- 172.21.15.17:0/1516452022 <== mon.2 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d40018070 con 0x7f4d481017e0 2023-07-14T11:59:36.356 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.372+0000 7f4d3f7fe700 1 -- 172.21.15.17:0/1516452022 <== mon.2 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d4000f040 con 0x7f4d481017e0 2023-07-14T11:59:36.356 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.372+0000 7f4d3f7fe700 1 -- 172.21.15.17:0/1516452022 <== mon.2 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4000cdf0 con 0x7f4d481017e0 2023-07-14T11:59:36.356 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.372+0000 7f4d503ff700 1 -- 172.21.15.17:0/1516452022 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d4819fbd0 con 0x7f4d481017e0 2023-07-14T11:59:36.356 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.372+0000 7f4d503ff700 1 -- 172.21.15.17:0/1516452022 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d481a0090 con 0x7f4d481017e0 2023-07-14T11:59:36.357 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.373+0000 7f4d503ff700 1 -- 172.21.15.17:0/1516452022 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d48106b70 con 0x7f4d481017e0 2023-07-14T11:59:36.357 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.375+0000 7f4d3f7fe700 1 -- 172.21.15.17:0/1516452022 <== mon.2 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4d40007500 con 0x7f4d481017e0 2023-07-14T11:59:36.357 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.376+0000 7f4d3f7fe700 1 --2- 172.21.15.17:0/1516452022 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f4d340a1150 0x7f4d340a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:36.357 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.376+0000 7f4d3f7fe700 1 -- 172.21.15.17:0/1516452022 <== mon.2 v2:172.21.15.93:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4d400cc110 con 0x7f4d481017e0 2023-07-14T11:59:36.358 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.376+0000 7f4d4e19b700 1 --2- 172.21.15.17:0/1516452022 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f4d340a1150 0x7f4d340a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:36.358 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.377+0000 7f4d4e19b700 1 --2- 172.21.15.17:0/1516452022 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f4d340a1150 0x7f4d340a3600 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4d4400b2e0 tx=0x7f4d44005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:36.360 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.380+0000 7f4d3f7fe700 1 -- 172.21.15.17:0/1516452022 <== mon.2 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4d40095cf0 con 0x7f4d481017e0 2023-07-14T11:59:36.548 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:36 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:36.549 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:36 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:36.549 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:36 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:36.549 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:36 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:59:36.550 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:36 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:59:36.550 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:36 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:59:36.550 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:36 smithi093 ceph-mon[128922]: Deploying daemon mds.b on smithi093 2023-07-14T11:59:36.550 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:36 smithi093 ceph-mon[128922]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:36.551 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:36 smithi093 ceph-mon[128922]: mds.? [v2:172.21.15.17:6840/3905201143,v1:172.21.15.17:6841/3905201143] up:boot 2023-07-14T11:59:36.551 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:36 smithi093 ceph-mon[128922]: fsmap 11 up:standby 2023-07-14T11:59:36.551 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:36 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-14T11:59:36.612 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.631+0000 7f4d503ff700 1 -- 172.21.15.17:0/1516452022 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d4802c850 con 0x7f4d481017e0 2023-07-14T11:59:36.613 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.632+0000 7f4d3f7fe700 1 -- 172.21.15.17:0/1516452022 <== mon.2 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+8816 (secure 0 0 0) 0x7f4d40099310 con 0x7f4d481017e0 2023-07-14T11:59:36.613 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:59:36.613 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/55384193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":55384193},{"type":"v1","addr":"172.21.15.17:6835","nonce":55384193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/424956554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":424956554},{"type":"v1","addr":"172.21.15.17:6837","nonce":424956554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/326319712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":326319712},{"type":"v1","addr":"172.21.15.93:6833","nonce":326319712}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/878913679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":878913679},{"type":"v1","addr":"172.21.15.97:6833","nonce":878913679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/2779420503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":2779420503},{"type":"v1","addr":"172.21.15.93:6835","nonce":2779420503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2740799393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2740799393},{"type":"v1","addr":"172.21.15.97:6835","nonce":2740799393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24509,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/1880673911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":1880673911},{"type":"v1","addr":"172.21.15.93:6837","nonce":1880673911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3684187131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3684187131},{"type":"v1","addr":"172.21.15.97:6837","nonce":3684187131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24533,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6839/2976493015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":2976493015},{"type":"v1","addr":"172.21.15.17:6839","nonce":2976493015}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24541,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/3267815331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3267815331},{"type":"v1","addr":"172.21.15.97:6839","nonce":3267815331}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24565,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6841/3905201143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6840","nonce":3905201143},{"type":"v1","addr":"172.21.15.17:6841","nonce":3905201143}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[]} 2023-07-14T11:59:36.615 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.634+0000 7f4d503ff700 1 -- 172.21.15.17:0/1516452022 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f4d340a1150 msgr2=0x7f4d340a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:36.615 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.634+0000 7f4d503ff700 1 --2- 172.21.15.17:0/1516452022 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f4d340a1150 0x7f4d340a3600 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4d4400b2e0 tx=0x7f4d44005fb0 comp rx=0 tx=0).stop 2023-07-14T11:59:36.616 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.634+0000 7f4d503ff700 1 -- 172.21.15.17:0/1516452022 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4d481017e0 msgr2=0x7f4d48072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:36.616 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.634+0000 7f4d503ff700 1 --2- 172.21.15.17:0/1516452022 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4d481017e0 0x7f4d48072e40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4d4000efc0 tx=0x7f4d4000c3a0 comp rx=0 tx=0).stop 2023-07-14T11:59:36.616 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.635+0000 7f4d503ff700 1 -- 172.21.15.17:0/1516452022 shutdown_connections 2023-07-14T11:59:36.616 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.635+0000 7f4d503ff700 1 --2- 172.21.15.17:0/1516452022 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f4d340a1150 0x7f4d340a3600 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:36.616 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.635+0000 7f4d503ff700 1 --2- 172.21.15.17:0/1516452022 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4d481017e0 0x7f4d48072e40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:36.616 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.635+0000 7f4d503ff700 1 --2- 172.21.15.17:0/1516452022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d480ffba0 0x7f4d480742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:36.616 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.635+0000 7f4d503ff700 1 --2- 172.21.15.17:0/1516452022 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d48100e30 0x7f4d48072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:36.617 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.635+0000 7f4d503ff700 1 -- 172.21.15.17:0/1516452022 >> 172.21.15.17:0/1516452022 conn(0x7f4d480fb1c0 msgr2=0x7f4d480fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:36.617 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.635+0000 7f4d503ff700 1 -- 172.21.15.17:0/1516452022 shutdown_connections 2023-07-14T11:59:36.617 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:36.635+0000 7f4d503ff700 1 -- 172.21.15.17:0/1516452022 wait complete. 2023-07-14T11:59:36.617 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 12 2023-07-14T11:59:36.632 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:36 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:36.633 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:36 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:36.633 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:36 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:36.633 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:36 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:59:36.633 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:36 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:59:36.633 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:36 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:59:36.633 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:36 smithi017 ceph-mon[126339]: Deploying daemon mds.b on smithi093 2023-07-14T11:59:36.634 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:36 smithi017 ceph-mon[126339]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:36.634 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:36 smithi017 ceph-mon[126339]: mds.? [v2:172.21.15.17:6840/3905201143,v1:172.21.15.17:6841/3905201143] up:boot 2023-07-14T11:59:36.634 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:36 smithi017 ceph-mon[126339]: fsmap 11 up:standby 2023-07-14T11:59:36.634 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:36 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-14T11:59:36.635 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:36 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:36.635 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:36 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:36.635 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:36 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:36.635 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:36 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T11:59:36.635 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:36 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T11:59:36.635 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:36 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:59:36.635 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:36 smithi097 ceph-mon[129313]: Deploying daemon mds.b on smithi093 2023-07-14T11:59:36.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:36 smithi097 ceph-mon[129313]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:36.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:36 smithi097 ceph-mon[129313]: mds.? [v2:172.21.15.17:6840/3905201143,v1:172.21.15.17:6841/3905201143] up:boot 2023-07-14T11:59:36.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:36 smithi097 ceph-mon[129313]: fsmap 11 up:standby 2023-07-14T11:59:36.636 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:36 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-14T11:59:37.111 INFO:journalctl@ceph.mds.b.smithi093.stdout:Jul 14 11:59:36 smithi093 systemd[1]: Starting Ceph mds.b for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T11:59:37.506 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:37 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/1516452022' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:59:37.506 INFO:journalctl@ceph.mds.b.smithi093.stdout:Jul 14 11:59:37 smithi093 podman[154464]: 2023-07-14T11:59:37.634 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:37 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/1516452022' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:59:37.644 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:37 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/1516452022' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:59:37.861 INFO:journalctl@ceph.mds.b.smithi093.stdout:Jul 14 11:59:37 smithi093 ceph-mds[154504]: -- 172.21.15.93:0/2278006543 <== mon.2 v2:172.21.15.93:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56119a58b1e0 con 0x56119a560c00 2023-07-14T11:59:37.861 INFO:journalctl@ceph.mds.b.smithi093.stdout:Jul 14 11:59:37 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-b[154480]: starting mds.b at 2023-07-14T11:59:37.861 INFO:journalctl@ceph.mds.b.smithi093.stdout:Jul 14 11:59:37 smithi093 bash[154464]: f90a320fdfd5f97fb9b0ef4cc85144aac271fdda42ca985fd088491941d83008 2023-07-14T11:59:37.861 INFO:journalctl@ceph.mds.b.smithi093.stdout:Jul 14 11:59:37 smithi093 systemd[1]: Started Ceph mds.b for b3c386cc-223c-11ee-9b33-001a4aab830c. 2023-07-14T11:59:38.638 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:59:38.851 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:38.870+0000 7f3b86fff700 1 -- 172.21.15.17:0/3492619384 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b800ffba0 msgr2=0x7f3b800fffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:38.851 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:38.870+0000 7f3b86fff700 1 --2- 172.21.15.17:0/3492619384 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b800ffba0 0x7f3b800fffb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3b74009990 tx=0x7f3b74009ca0 comp rx=0 tx=0).stop 2023-07-14T11:59:38.851 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:38.870+0000 7f3b86fff700 1 -- 172.21.15.17:0/3492619384 shutdown_connections 2023-07-14T11:59:38.852 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:38.870+0000 7f3b86fff700 1 --2- 172.21.15.17:0/3492619384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b801017e0 0x7f3b80108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:38.852 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:38.870+0000 7f3b86fff700 1 --2- 172.21.15.17:0/3492619384 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b800ffba0 0x7f3b800fffb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:38.852 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:38.870+0000 7f3b86fff700 1 --2- 172.21.15.17:0/3492619384 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b80100e30 0x7f3b801012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:38.852 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:38.870+0000 7f3b86fff700 1 -- 172.21.15.17:0/3492619384 >> 172.21.15.17:0/3492619384 conn(0x7f3b800fb1c0 msgr2=0x7f3b800fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:38.852 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:38.870+0000 7f3b86fff700 1 -- 172.21.15.17:0/3492619384 shutdown_connections 2023-07-14T11:59:38.852 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:38.870+0000 7f3b86fff700 1 -- 172.21.15.17:0/3492619384 wait complete. 2023-07-14T11:59:38.854 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:38.871+0000 7f3b86fff700 1 Processor -- start 2023-07-14T11:59:38.854 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:38.872+0000 7f3b86fff700 1 -- start start 2023-07-14T11:59:38.854 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:38.872+0000 7f3b86fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b800ffba0 0x7f3b800742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:38.854 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:38.872+0000 7f3b86fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b80100e30 0x7f3b80072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:38.854 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:38.872+0000 7f3b86fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b801017e0 0x7f3b80072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:38.855 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:38.872+0000 7f3b86fff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b800735e0 con 0x7f3b80100e30 2023-07-14T11:59:38.855 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:38.872+0000 7f3b86fff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b80073750 con 0x7f3b801017e0 2023-07-14T11:59:38.855 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:38.872+0000 7f3b86fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8019f9a0 con 0x7f3b800ffba0 2023-07-14T11:59:38.855 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:38.872+0000 7f3b7ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b80100e30 0x7f3b80072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:38.855 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:38.872+0000 7f3b7ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b80100e30 0x7f3b80072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51766/0 (socket says 172.21.15.17:51766) 2023-07-14T11:59:38.855 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:38.872+0000 7f3b7ffff700 1 -- 172.21.15.17:0/802007767 learned_addr learned my addr 172.21.15.17:0/802007767 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:59:38.855 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:38.872+0000 7f3b8559c700 1 --2- 172.21.15.17:0/802007767 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b801017e0 0x7f3b80072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:38.856 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:38.872+0000 7f3b84d9b700 1 --2- 172.21.15.17:0/802007767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b800ffba0 0x7f3b800742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:38.856 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:38.873+0000 7f3b8559c700 1 -- 172.21.15.17:0/802007767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b800ffba0 msgr2=0x7f3b800742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:38.856 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:38.873+0000 7f3b8559c700 1 --2- 172.21.15.17:0/802007767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b800ffba0 0x7f3b800742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:38.856 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:38.873+0000 7f3b8559c700 1 -- 172.21.15.17:0/802007767 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b80100e30 msgr2=0x7f3b80072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:38.856 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:38.873+0000 7f3b8559c700 1 --2- 172.21.15.17:0/802007767 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b80100e30 0x7f3b80072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:38.856 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:38.873+0000 7f3b8559c700 1 -- 172.21.15.17:0/802007767 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b6c009520 con 0x7f3b801017e0 2023-07-14T11:59:38.856 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:38.873+0000 7f3b7ffff700 1 --2- 172.21.15.17:0/802007767 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b80100e30 0x7f3b80072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:59:38.856 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:38.873+0000 7f3b8559c700 1 --2- 172.21.15.17:0/802007767 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b801017e0 0x7f3b80072e40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3b7800efc0 tx=0x7f3b7800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:38.857 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:38.873+0000 7f3b7dffb700 1 -- 172.21.15.17:0/802007767 <== mon.2 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b78018070 con 0x7f3b801017e0 2023-07-14T11:59:38.857 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:38.873+0000 7f3b7dffb700 1 -- 172.21.15.17:0/802007767 <== mon.2 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b7800f040 con 0x7f3b801017e0 2023-07-14T11:59:38.857 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:38.873+0000 7f3b86fff700 1 -- 172.21.15.17:0/802007767 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b740095f0 con 0x7f3b801017e0 2023-07-14T11:59:38.857 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:38.873+0000 7f3b7dffb700 1 -- 172.21.15.17:0/802007767 <== mon.2 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7800cd40 con 0x7f3b801017e0 2023-07-14T11:59:38.857 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:38.873+0000 7f3b86fff700 1 -- 172.21.15.17:0/802007767 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b8019ffa0 con 0x7f3b801017e0 2023-07-14T11:59:38.858 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:38.874+0000 7f3b86fff700 1 -- 172.21.15.17:0/802007767 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b80106bc0 con 0x7f3b801017e0 2023-07-14T11:59:38.858 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:38.876+0000 7f3b7dffb700 1 -- 172.21.15.17:0/802007767 <== mon.2 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3b78003a00 con 0x7f3b801017e0 2023-07-14T11:59:38.858 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:38.878+0000 7f3b7dffb700 1 --2- 172.21.15.17:0/802007767 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3b700a1150 0x7f3b700a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:38.859 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:38.878+0000 7f3b7dffb700 1 -- 172.21.15.17:0/802007767 <== mon.2 v2:172.21.15.93:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3b780cca60 con 0x7f3b801017e0 2023-07-14T11:59:38.859 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:38.878+0000 7f3b84d9b700 1 --2- 172.21.15.17:0/802007767 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3b700a1150 0x7f3b700a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:38.860 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:38.879+0000 7f3b84d9b700 1 --2- 172.21.15.17:0/802007767 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3b700a1150 0x7f3b700a3600 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3b7400b2e0 tx=0x7f3b74005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:38.860 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:38.880+0000 7f3b7dffb700 1 -- 172.21.15.17:0/802007767 <== mon.2 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3b78096640 con 0x7f3b801017e0 2023-07-14T11:59:38.861 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:38 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:38.861 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:38 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:38.861 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:38 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:38.861 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:38 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:38.861 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:38 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:38.861 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:38 smithi093 ceph-mon[128922]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:38.862 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:38 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:59:38.862 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:38 smithi093 ceph-mon[128922]: mds.? [v2:172.21.15.93:6838/1281342428,v1:172.21.15.93:6839/1281342428] up:boot 2023-07-14T11:59:38.862 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:38 smithi093 ceph-mon[128922]: fsmap 12 up:standby 2023-07-14T11:59:38.862 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:38 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-14T11:59:38.884 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:38 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:38.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:38 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:38.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:38 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:38.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:38 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:38.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:38 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:38.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:38 smithi097 ceph-mon[129313]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:38.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:38 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:59:38.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:38 smithi097 ceph-mon[129313]: mds.? [v2:172.21.15.93:6838/1281342428,v1:172.21.15.93:6839/1281342428] up:boot 2023-07-14T11:59:38.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:38 smithi097 ceph-mon[129313]: fsmap 12 up:standby 2023-07-14T11:59:38.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:38 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-14T11:59:38.894 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:38 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:38.894 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:38 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:38.894 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:38 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:38.894 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:38 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:38.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:38 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:38.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:38 smithi017 ceph-mon[126339]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:38.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:38 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:59:38.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:38 smithi017 ceph-mon[126339]: mds.? [v2:172.21.15.93:6838/1281342428,v1:172.21.15.93:6839/1281342428] up:boot 2023-07-14T11:59:38.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:38 smithi017 ceph-mon[126339]: fsmap 12 up:standby 2023-07-14T11:59:38.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:38 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-14T11:59:39.135 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.155+0000 7f3b86fff700 1 -- 172.21.15.17:0/802007767 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3b8002c850 con 0x7f3b801017e0 2023-07-14T11:59:39.136 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.156+0000 7f3b7dffb700 1 -- 172.21.15.17:0/802007767 <== mon.2 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9574 (secure 0 0 0) 0x7f3b78099c60 con 0x7f3b801017e0 2023-07-14T11:59:39.136 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:59:39.137 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/55384193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":55384193},{"type":"v1","addr":"172.21.15.17:6835","nonce":55384193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/424956554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":424956554},{"type":"v1","addr":"172.21.15.17:6837","nonce":424956554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/326319712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":326319712},{"type":"v1","addr":"172.21.15.93:6833","nonce":326319712}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/878913679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":878913679},{"type":"v1","addr":"172.21.15.97:6833","nonce":878913679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/2779420503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":2779420503},{"type":"v1","addr":"172.21.15.93:6835","nonce":2779420503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2740799393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2740799393},{"type":"v1","addr":"172.21.15.97:6835","nonce":2740799393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24509,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/1880673911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":1880673911},{"type":"v1","addr":"172.21.15.93:6837","nonce":1880673911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3684187131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3684187131},{"type":"v1","addr":"172.21.15.97:6837","nonce":3684187131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24533,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6839/2976493015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":2976493015},{"type":"v1","addr":"172.21.15.17:6839","nonce":2976493015}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24541,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/3267815331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3267815331},{"type":"v1","addr":"172.21.15.97:6839","nonce":3267815331}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6839/1281342428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":1281342428},{"type":"v1","addr":"172.21.15.93:6839","nonce":1281342428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24565,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6841/3905201143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6840","nonce":3905201143},{"type":"v1","addr":"172.21.15.17:6841","nonce":3905201143}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[]} 2023-07-14T11:59:39.139 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.158+0000 7f3b86fff700 1 -- 172.21.15.17:0/802007767 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3b700a1150 msgr2=0x7f3b700a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:39.139 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.158+0000 7f3b86fff700 1 --2- 172.21.15.17:0/802007767 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3b700a1150 0x7f3b700a3600 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3b7400b2e0 tx=0x7f3b74005fb0 comp rx=0 tx=0).stop 2023-07-14T11:59:39.139 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.158+0000 7f3b86fff700 1 -- 172.21.15.17:0/802007767 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b801017e0 msgr2=0x7f3b80072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:39.139 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.158+0000 7f3b86fff700 1 --2- 172.21.15.17:0/802007767 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b801017e0 0x7f3b80072e40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3b7800efc0 tx=0x7f3b7800c3a0 comp rx=0 tx=0).stop 2023-07-14T11:59:39.140 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.158+0000 7f3b86fff700 1 -- 172.21.15.17:0/802007767 shutdown_connections 2023-07-14T11:59:39.140 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.158+0000 7f3b86fff700 1 --2- 172.21.15.17:0/802007767 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3b700a1150 0x7f3b700a3600 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:39.140 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.158+0000 7f3b86fff700 1 --2- 172.21.15.17:0/802007767 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b801017e0 0x7f3b80072e40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:39.140 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.158+0000 7f3b86fff700 1 --2- 172.21.15.17:0/802007767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b800ffba0 0x7f3b800742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:39.140 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.158+0000 7f3b86fff700 1 --2- 172.21.15.17:0/802007767 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b80100e30 0x7f3b80072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:39.140 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.158+0000 7f3b86fff700 1 -- 172.21.15.17:0/802007767 >> 172.21.15.17:0/802007767 conn(0x7f3b800fb1c0 msgr2=0x7f3b800fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:39.140 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.158+0000 7f3b86fff700 1 -- 172.21.15.17:0/802007767 shutdown_connections 2023-07-14T11:59:39.141 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.158+0000 7f3b86fff700 1 -- 172.21.15.17:0/802007767 wait complete. 2023-07-14T11:59:39.141 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 13 2023-07-14T11:59:39.158 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:59:39.372 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.391+0000 7fcf0c1ff700 1 -- 172.21.15.17:0/2918076011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf04072ee0 msgr2=0x7fcf04073350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:39.372 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.391+0000 7fcf0c1ff700 1 --2- 172.21.15.17:0/2918076011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf04072ee0 0x7fcf04073350 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcef4009990 tx=0x7fcef4009ca0 comp rx=0 tx=0).stop 2023-07-14T11:59:39.372 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.391+0000 7fcf0c1ff700 1 -- 172.21.15.17:0/2918076011 shutdown_connections 2023-07-14T11:59:39.372 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.391+0000 7fcf0c1ff700 1 --2- 172.21.15.17:0/2918076011 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcf040744b0 0x7fcf040729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:39.372 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.391+0000 7fcf0c1ff700 1 --2- 172.21.15.17:0/2918076011 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf04103620 0x7fcf0410e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:39.373 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.391+0000 7fcf0c1ff700 1 --2- 172.21.15.17:0/2918076011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf04072ee0 0x7fcf04073350 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:39.373 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.391+0000 7fcf0c1ff700 1 -- 172.21.15.17:0/2918076011 >> 172.21.15.17:0/2918076011 conn(0x7fcf040fb1d0 msgr2=0x7fcf040fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:39.373 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.391+0000 7fcf0c1ff700 1 -- 172.21.15.17:0/2918076011 shutdown_connections 2023-07-14T11:59:39.373 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.391+0000 7fcf0c1ff700 1 -- 172.21.15.17:0/2918076011 wait complete. 2023-07-14T11:59:39.375 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.392+0000 7fcf0c1ff700 1 Processor -- start 2023-07-14T11:59:39.375 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.392+0000 7fcf0c1ff700 1 -- start start 2023-07-14T11:59:39.375 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.392+0000 7fcf0c1ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf04072ee0 0x7fcf0419bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:39.375 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.392+0000 7fcf0c1ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcf040744b0 0x7fcf0419c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:39.375 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.392+0000 7fcf0c1ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf04103620 0x7fcf041a34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:39.375 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.392+0000 7fcf0c1ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf041a3a70 con 0x7fcf04103620 2023-07-14T11:59:39.376 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.392+0000 7fcf0c1ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf041a3bb0 con 0x7fcf040744b0 2023-07-14T11:59:39.376 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.392+0000 7fcf0c1ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf041a3d40 con 0x7fcf04072ee0 2023-07-14T11:59:39.376 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.393+0000 7fcf0979a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcf040744b0 0x7fcf0419c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:39.376 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.393+0000 7fcf0a79c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf04103620 0x7fcf041a34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:39.376 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.393+0000 7fcf0979a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcf040744b0 0x7fcf0419c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.17:40510/0 (socket says 172.21.15.17:40510) 2023-07-14T11:59:39.376 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.393+0000 7fcf0979a700 1 -- 172.21.15.17:0/1083841047 learned_addr learned my addr 172.21.15.17:0/1083841047 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:59:39.376 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.393+0000 7fcf09f9b700 1 --2- 172.21.15.17:0/1083841047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf04072ee0 0x7fcf0419bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:39.376 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.393+0000 7fcf0979a700 1 -- 172.21.15.17:0/1083841047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf04072ee0 msgr2=0x7fcf0419bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:39.377 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.393+0000 7fcf0979a700 1 --2- 172.21.15.17:0/1083841047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf04072ee0 0x7fcf0419bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:39.377 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.393+0000 7fcf0979a700 1 -- 172.21.15.17:0/1083841047 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf04103620 msgr2=0x7fcf041a34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:39.377 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.393+0000 7fcf0979a700 1 --2- 172.21.15.17:0/1083841047 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf04103620 0x7fcf041a34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:39.377 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.393+0000 7fcf0979a700 1 -- 172.21.15.17:0/1083841047 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef40095f0 con 0x7fcf040744b0 2023-07-14T11:59:39.377 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.393+0000 7fcf0979a700 1 --2- 172.21.15.17:0/1083841047 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcf040744b0 0x7fcf0419c430 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcef4004740 tx=0x7fcef4009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:39.377 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.394+0000 7fcefaffd700 1 -- 172.21.15.17:0/1083841047 <== mon.2 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef401d070 con 0x7fcf040744b0 2023-07-14T11:59:39.378 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.394+0000 7fcf0c1ff700 1 -- 172.21.15.17:0/1083841047 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf041a3fc0 con 0x7fcf040744b0 2023-07-14T11:59:39.378 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.394+0000 7fcefaffd700 1 -- 172.21.15.17:0/1083841047 <== mon.2 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcef4004830 con 0x7fcf040744b0 2023-07-14T11:59:39.378 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.394+0000 7fcefaffd700 1 -- 172.21.15.17:0/1083841047 <== mon.2 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef40053c0 con 0x7fcf040744b0 2023-07-14T11:59:39.378 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.394+0000 7fcf0c1ff700 1 -- 172.21.15.17:0/1083841047 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf041a4330 con 0x7fcf040744b0 2023-07-14T11:59:39.379 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.395+0000 7fcf0c1ff700 1 -- 172.21.15.17:0/1083841047 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf04101d70 con 0x7fcf040744b0 2023-07-14T11:59:39.379 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.397+0000 7fcefaffd700 1 -- 172.21.15.17:0/1083841047 <== mon.2 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fcef4005580 con 0x7fcf040744b0 2023-07-14T11:59:39.379 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.399+0000 7fcefaffd700 1 --2- 172.21.15.17:0/1083841047 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fcef00a1150 0x7fcef00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:39.380 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.399+0000 7fcefaffd700 1 -- 172.21.15.17:0/1083841047 <== mon.2 v2:172.21.15.93:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fcef40cd650 con 0x7fcf040744b0 2023-07-14T11:59:39.380 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.399+0000 7fcf09f9b700 1 --2- 172.21.15.17:0/1083841047 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fcef00a1150 0x7fcef00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:39.380 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.400+0000 7fcf09f9b700 1 --2- 172.21.15.17:0/1083841047 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fcef00a1150 0x7fcef00a3600 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcf040741d0 tx=0x7fcf00009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:39.382 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.402+0000 7fcefaffd700 1 -- 172.21.15.17:0/1083841047 <== mon.2 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fcef405a790 con 0x7fcf040744b0 2023-07-14T11:59:39.599 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.619+0000 7fcf0c1ff700 1 -- 172.21.15.17:0/1083841047 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcf040611c0 con 0x7fcf040744b0 2023-07-14T11:59:39.601 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.620+0000 7fcefaffd700 1 -- 172.21.15.17:0/1083841047 <== mon.2 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+18328 (secure 0 0 0) 0x7fcef4097230 con 0x7fcf040744b0 2023-07-14T11:59:39.601 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:59:39.602 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":80,"fsid":"b3c386cc-223c-11ee-9b33-001a4aab830c","created":"2023-07-14T11:51:59.992960+0000","modified":"2023-07-14T11:58:56.048722+0000","last_up_change":"2023-07-14T11:58:55.037769+0000","last_in_change":"2023-07-14T11:58:37.114870+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:55:28.237002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57003438-a83c-4c8e-a029-56414740895c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6803","nonce":3588366384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6805","nonce":3588366384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6809","nonce":3588366384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6807","nonce":3588366384}]},"public_addr":"172.21.15.17:6803/3588366384","cluster_addr":"172.21.15.17:6805/3588366384","heartbeat_back_addr":"172.21.15.17:6809/3588366384","heartbeat_front_addr":"172.21.15.17:6807/3588366384","state":["exists","up"]},{"osd":1,"uuid":"f56b6337-d8a9-4411-b9d8-b6863dc98556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6801","nonce":3147203064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6803","nonce":3147203064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6807","nonce":3147203064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6805","nonce":3147203064}]},"public_addr":"172.21.15.93:6801/3147203064","cluster_addr":"172.21.15.93:6803/3147203064","heartbeat_back_addr":"172.21.15.93:6807/3147203064","heartbeat_front_addr":"172.21.15.93:6805/3147203064","state":["exists","up"]},{"osd":2,"uuid":"fc34bb7f-551f-46e1-99c0-59924d6c1346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6801","nonce":1877269815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6803","nonce":1877269815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6807","nonce":1877269815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6805","nonce":1877269815}]},"public_addr":"172.21.15.97:6801/1877269815","cluster_addr":"172.21.15.97:6803/1877269815","heartbeat_back_addr":"172.21.15.97:6807/1877269815","heartbeat_front_addr":"172.21.15.97:6805/1877269815","state":["exists","up"]},{"osd":3,"uuid":"37a1583c-4fb8-412a-a0c8-f83b1ff26a91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6811","nonce":2784619821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6813","nonce":2784619821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6817","nonce":2784619821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6815","nonce":2784619821}]},"public_addr":"172.21.15.17:6811/2784619821","cluster_addr":"172.21.15.17:6813/2784619821","heartbeat_back_addr":"172.21.15.17:6817/2784619821","heartbeat_front_addr":"172.21.15.17:6815/2784619821","state":["exists","up"]},{"osd":4,"uuid":"f4b9f377-d519-4b14-ac4a-f1debaeed9f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6809","nonce":3151801173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6811","nonce":3151801173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6815","nonce":3151801173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6813","nonce":3151801173}]},"public_addr":"172.21.15.93:6809/3151801173","cluster_addr":"172.21.15.93:6811/3151801173","heartbeat_back_addr":"172.21.15.93:6815/3151801173","heartbeat_front_addr":"172.21.15.93:6813/3151801173","state":["exists","up"]},{"osd":5,"uuid":"2d2af354-0535-4ef9-b166-3ef219498db8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6809","nonce":2184413924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6811","nonce":2184413924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6815","nonce":2184413924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6813","nonce":2184413924}]},"public_addr":"172.21.15.97:6809/2184413924","cluster_addr":"172.21.15.97:6811/2184413924","heartbeat_back_addr":"172.21.15.97:6815/2184413924","heartbeat_front_addr":"172.21.15.97:6813/2184413924","state":["exists","up"]},{"osd":6,"uuid":"9cdf0c4b-62dc-4168-864e-a9d1650a9ea8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6819","nonce":50486581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6821","nonce":50486581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6825","nonce":50486581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6823","nonce":50486581}]},"public_addr":"172.21.15.17:6819/50486581","cluster_addr":"172.21.15.17:6821/50486581","heartbeat_back_addr":"172.21.15.17:6825/50486581","heartbeat_front_addr":"172.21.15.17:6823/50486581","state":["exists","up"]},{"osd":7,"uuid":"09f60823-5491-45cd-9324-6ae0d56ce262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6817","nonce":2596912392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6819","nonce":2596912392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6823","nonce":2596912392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6821","nonce":2596912392}]},"public_addr":"172.21.15.93:6817/2596912392","cluster_addr":"172.21.15.93:6819/2596912392","heartbeat_back_addr":"172.21.15.93:6823/2596912392","heartbeat_front_addr":"172.21.15.93:6821/2596912392","state":["exists","up"]},{"osd":8,"uuid":"8c5771c9-718e-4313-a2b6-5e1c37c53f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6817","nonce":1965492859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6819","nonce":1965492859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6823","nonce":1965492859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6821","nonce":1965492859}]},"public_addr":"172.21.15.97:6817/1965492859","cluster_addr":"172.21.15.97:6819/1965492859","heartbeat_back_addr":"172.21.15.97:6823/1965492859","heartbeat_front_addr":"172.21.15.97:6821/1965492859","state":["exists","up"]},{"osd":9,"uuid":"1197b8a3-c09f-4c57-aba1-9e988a1620b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6827","nonce":2512645700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6829","nonce":2512645700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6833","nonce":2512645700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6831","nonce":2512645700}]},"public_addr":"172.21.15.17:6827/2512645700","cluster_addr":"172.21.15.17:6829/2512645700","heartbeat_back_addr":"172.21.15.17:6833/2512645700","heartbeat_front_addr":"172.21.15.17:6831/2512645700","state":["exists","up"]},{"osd":10,"uuid":"f99918c2-cdd8-4a7a-be2e-d1d6e0f828a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6825","nonce":264429558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6827","nonce":264429558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6831","nonce":264429558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6829","nonce":264429558}]},"public_addr":"172.21.15.93:6825/264429558","cluster_addr":"172.21.15.93:6827/264429558","heartbeat_back_addr":"172.21.15.93:6831/264429558","heartbeat_front_addr":"172.21.15.93:6829/264429558","state":["exists","up"]},{"osd":11,"uuid":"6bbb5b79-8d4d-441f-9583-203df6c78cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6825","nonce":3389913754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6827","nonce":3389913754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6831","nonce":3389913754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6829","nonce":3389913754}]},"public_addr":"172.21.15.97:6825/3389913754","cluster_addr":"172.21.15.97:6827/3389913754","heartbeat_back_addr":"172.21.15.97:6831/3389913754","heartbeat_front_addr":"172.21.15.97:6829/3389913754","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:54:41.079618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:03.207899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:24.136961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:50.392609+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:56:12.136237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:56:33.212314+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:00.243646+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:22.059234+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:43.232399+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:10.043896+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:31.910780+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:53.158983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/2084033276":"2023-07-15T11:53:05.639597+0000","172.21.15.17:6801/1059284716":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/3651785636":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/1373538024":"2023-07-15T11:52:37.887946+0000","172.21.15.17:0/3560395020":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6800/1059284716":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/4262912861":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/2258375202":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6800/2804227742":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6801/2804227742":"2023-07-15T11:52:37.887946+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:59:39.603 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.623+0000 7fcf0c1ff700 1 -- 172.21.15.17:0/1083841047 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fcef00a1150 msgr2=0x7fcef00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:39.603 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.623+0000 7fcf0c1ff700 1 --2- 172.21.15.17:0/1083841047 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fcef00a1150 0x7fcef00a3600 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcf040741d0 tx=0x7fcf00009260 comp rx=0 tx=0).stop 2023-07-14T11:59:39.604 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.623+0000 7fcf0c1ff700 1 -- 172.21.15.17:0/1083841047 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcf040744b0 msgr2=0x7fcf0419c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:39.604 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.623+0000 7fcf0c1ff700 1 --2- 172.21.15.17:0/1083841047 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcf040744b0 0x7fcf0419c430 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcef4004740 tx=0x7fcef4009580 comp rx=0 tx=0).stop 2023-07-14T11:59:39.604 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.623+0000 7fcf0c1ff700 1 -- 172.21.15.17:0/1083841047 shutdown_connections 2023-07-14T11:59:39.604 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.623+0000 7fcf0c1ff700 1 --2- 172.21.15.17:0/1083841047 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fcef00a1150 0x7fcef00a3600 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:39.604 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.623+0000 7fcf0c1ff700 1 --2- 172.21.15.17:0/1083841047 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf04103620 0x7fcf041a34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:39.604 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.623+0000 7fcf0c1ff700 1 --2- 172.21.15.17:0/1083841047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf04072ee0 0x7fcf0419bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:39.605 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.623+0000 7fcf0c1ff700 1 --2- 172.21.15.17:0/1083841047 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcf040744b0 0x7fcf0419c430 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:39.605 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.623+0000 7fcf0c1ff700 1 -- 172.21.15.17:0/1083841047 >> 172.21.15.17:0/1083841047 conn(0x7fcf040fb1d0 msgr2=0x7fcf040fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:39.605 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.623+0000 7fcf0c1ff700 1 -- 172.21.15.17:0/1083841047 shutdown_connections 2023-07-14T11:59:39.605 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.623+0000 7fcf0c1ff700 1 -- 172.21.15.17:0/1083841047 wait complete. 2023-07-14T11:59:39.627 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:55:28.237002+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T11:59:39.628 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T11:59:39.847 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.866+0000 7fbdd21ff700 1 -- 172.21.15.17:0/3534730570 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbdcc0ffbc0 msgr2=0x7fbdcc0fffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:39.847 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.866+0000 7fbdd21ff700 1 --2- 172.21.15.17:0/3534730570 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbdcc0ffbc0 0x7fbdcc0fffd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc009990 tx=0x7fbdbc009ca0 comp rx=0 tx=0).stop 2023-07-14T11:59:39.848 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.866+0000 7fbdd21ff700 1 -- 172.21.15.17:0/3534730570 shutdown_connections 2023-07-14T11:59:39.848 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.866+0000 7fbdd21ff700 1 --2- 172.21.15.17:0/3534730570 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbdcc101800 0x7fbdcc1080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:39.848 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.866+0000 7fbdd21ff700 1 --2- 172.21.15.17:0/3534730570 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbdcc0ffbc0 0x7fbdcc0fffd0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:39.848 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.866+0000 7fbdd21ff700 1 --2- 172.21.15.17:0/3534730570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdcc100e50 0x7fbdcc1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:39.848 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.866+0000 7fbdd21ff700 1 -- 172.21.15.17:0/3534730570 >> 172.21.15.17:0/3534730570 conn(0x7fbdcc0fb200 msgr2=0x7fbdcc0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:39.848 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.866+0000 7fbdd21ff700 1 -- 172.21.15.17:0/3534730570 shutdown_connections 2023-07-14T11:59:39.849 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.866+0000 7fbdd21ff700 1 -- 172.21.15.17:0/3534730570 wait complete. 2023-07-14T11:59:39.850 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.867+0000 7fbdd21ff700 1 Processor -- start 2023-07-14T11:59:39.850 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.867+0000 7fbdd21ff700 1 -- start start 2023-07-14T11:59:39.850 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.868+0000 7fbdd21ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbdcc0ffbc0 0x7fbdcc0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:39.851 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.868+0000 7fbdd21ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdcc100e50 0x7fbdcc072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:39.851 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.868+0000 7fbdd21ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbdcc101800 0x7fbdcc072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:39.851 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.868+0000 7fbdd21ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdcc0735e0 con 0x7fbdcc0ffbc0 2023-07-14T11:59:39.851 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.868+0000 7fbdd21ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdcc073750 con 0x7fbdcc101800 2023-07-14T11:59:39.851 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.868+0000 7fbdd21ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdcc19f940 con 0x7fbdcc100e50 2023-07-14T11:59:39.851 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.868+0000 7fbdcb7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbdcc0ffbc0 0x7fbdcc0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:39.851 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.868+0000 7fbdcb7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbdcc0ffbc0 0x7fbdcc0742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51798/0 (socket says 172.21.15.17:51798) 2023-07-14T11:59:39.852 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.868+0000 7fbdcb7fe700 1 -- 172.21.15.17:0/3671569470 learned_addr learned my addr 172.21.15.17:0/3671569470 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:59:39.852 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.868+0000 7fbdcbfff700 1 --2- 172.21.15.17:0/3671569470 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbdcc101800 0x7fbdcc072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:39.852 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.868+0000 7fbdcaffd700 1 --2- 172.21.15.17:0/3671569470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdcc100e50 0x7fbdcc072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:39.852 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.868+0000 7fbdcb7fe700 1 -- 172.21.15.17:0/3671569470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdcc100e50 msgr2=0x7fbdcc072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:39.852 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.868+0000 7fbdcb7fe700 1 --2- 172.21.15.17:0/3671569470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdcc100e50 0x7fbdcc072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:39.852 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.868+0000 7fbdcb7fe700 1 -- 172.21.15.17:0/3671569470 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbdcc101800 msgr2=0x7fbdcc072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:39.852 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.868+0000 7fbdcb7fe700 1 --2- 172.21.15.17:0/3671569470 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbdcc101800 0x7fbdcc072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:39.853 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.868+0000 7fbdcb7fe700 1 -- 172.21.15.17:0/3671569470 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdbc0095f0 con 0x7fbdcc0ffbc0 2023-07-14T11:59:39.853 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.868+0000 7fbdcbfff700 1 --2- 172.21.15.17:0/3671569470 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbdcc101800 0x7fbdcc072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:59:39.853 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.869+0000 7fbdcb7fe700 1 --2- 172.21.15.17:0/3671569470 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbdcc0ffbc0 0x7fbdcc0742c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc00b680 tx=0x7fbdbc005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:39.853 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.869+0000 7fbdc8ff9700 1 -- 172.21.15.17:0/3671569470 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc01d070 con 0x7fbdcc0ffbc0 2023-07-14T11:59:39.853 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.869+0000 7fbdd21ff700 1 -- 172.21.15.17:0/3671569470 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdcc19fbc0 con 0x7fbdcc0ffbc0 2023-07-14T11:59:39.853 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.869+0000 7fbdc8ff9700 1 -- 172.21.15.17:0/3671569470 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbdbc022470 con 0x7fbdcc0ffbc0 2023-07-14T11:59:39.853 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.869+0000 7fbdc8ff9700 1 -- 172.21.15.17:0/3671569470 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc017650 con 0x7fbdcc0ffbc0 2023-07-14T11:59:39.853 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.869+0000 7fbdd21ff700 1 -- 172.21.15.17:0/3671569470 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdcc19ffe0 con 0x7fbdcc0ffbc0 2023-07-14T11:59:39.854 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.870+0000 7fbdd21ff700 1 -- 172.21.15.17:0/3671569470 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdcc1071a0 con 0x7fbdcc0ffbc0 2023-07-14T11:59:39.854 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.872+0000 7fbdc8ff9700 1 -- 172.21.15.17:0/3671569470 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fbdbc0225e0 con 0x7fbdcc0ffbc0 2023-07-14T11:59:39.855 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.873+0000 7fbdc8ff9700 1 --2- 172.21.15.17:0/3671569470 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fbdb40a1150 0x7fbdb40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:39.855 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.873+0000 7fbdc8ff9700 1 -- 172.21.15.17:0/3671569470 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbdbc0cd9c0 con 0x7fbdcc0ffbc0 2023-07-14T11:59:39.855 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.874+0000 7fbdcaffd700 1 --2- 172.21.15.17:0/3671569470 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fbdb40a1150 0x7fbdb40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:39.855 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.874+0000 7fbdcaffd700 1 --2- 172.21.15.17:0/3671569470 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fbdb40a1150 0x7fbdb40a3600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0009c60 tx=0x7fbdc0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:39.858 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:39.878+0000 7fbdc8ff9700 1 -- 172.21.15.17:0/3671569470 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fbdbc0975a0 con 0x7fbdcc0ffbc0 2023-07-14T11:59:39.884 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:39 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/802007767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:59:39.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:39 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/1083841047' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:59:39.894 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:39 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/802007767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:59:39.894 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:39 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/1083841047' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:59:40.071 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.089+0000 7fbdd21ff700 1 -- 172.21.15.17:0/3671569470 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbdcc061120 con 0x7fbdcc0ffbc0 2023-07-14T11:59:40.071 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.090+0000 7fbdc8ff9700 1 -- 172.21.15.17:0/3671569470 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbdbc011110 con 0x7fbdcc0ffbc0 2023-07-14T11:59:40.072 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2023-07-14T11:59:40.073 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.092+0000 7fbdd21ff700 1 -- 172.21.15.17:0/3671569470 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fbdb40a1150 msgr2=0x7fbdb40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:40.073 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.092+0000 7fbdd21ff700 1 --2- 172.21.15.17:0/3671569470 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fbdb40a1150 0x7fbdb40a3600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0009c60 tx=0x7fbdc0009260 comp rx=0 tx=0).stop 2023-07-14T11:59:40.073 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.092+0000 7fbdd21ff700 1 -- 172.21.15.17:0/3671569470 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbdcc0ffbc0 msgr2=0x7fbdcc0742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:40.074 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.092+0000 7fbdd21ff700 1 --2- 172.21.15.17:0/3671569470 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbdcc0ffbc0 0x7fbdcc0742c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc00b680 tx=0x7fbdbc005740 comp rx=0 tx=0).stop 2023-07-14T11:59:40.074 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.092+0000 7fbdd21ff700 1 -- 172.21.15.17:0/3671569470 shutdown_connections 2023-07-14T11:59:40.074 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.092+0000 7fbdd21ff700 1 --2- 172.21.15.17:0/3671569470 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fbdb40a1150 0x7fbdb40a3600 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:40.074 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.092+0000 7fbdd21ff700 1 --2- 172.21.15.17:0/3671569470 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbdcc101800 0x7fbdcc072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:40.074 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.092+0000 7fbdd21ff700 1 --2- 172.21.15.17:0/3671569470 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbdcc0ffbc0 0x7fbdcc0742c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:40.074 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.092+0000 7fbdd21ff700 1 --2- 172.21.15.17:0/3671569470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdcc100e50 0x7fbdcc072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:40.074 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.092+0000 7fbdd21ff700 1 -- 172.21.15.17:0/3671569470 >> 172.21.15.17:0/3671569470 conn(0x7fbdcc0fb200 msgr2=0x7fbdcc1042e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:40.074 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.093+0000 7fbdd21ff700 1 -- 172.21.15.17:0/3671569470 shutdown_connections 2023-07-14T11:59:40.075 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.093+0000 7fbdd21ff700 1 -- 172.21.15.17:0/3671569470 wait complete. 2023-07-14T11:59:40.092 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-14T11:59:40.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:39 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/802007767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:59:40.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:39 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/1083841047' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:59:40.303 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.322+0000 7f5648bff700 1 -- 172.21.15.17:0/1505630279 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56440ffba0 msgr2=0x7f56440fffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:40.303 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.322+0000 7f5648bff700 1 --2- 172.21.15.17:0/1505630279 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56440ffba0 0x7f56440fffb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5634009990 tx=0x7f5634009ca0 comp rx=0 tx=0).stop 2023-07-14T11:59:40.303 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.322+0000 7f5648bff700 1 -- 172.21.15.17:0/1505630279 shutdown_connections 2023-07-14T11:59:40.304 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.322+0000 7f5648bff700 1 --2- 172.21.15.17:0/1505630279 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56440ffba0 0x7f56440fffb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:40.304 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.322+0000 7f5648bff700 1 --2- 172.21.15.17:0/1505630279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56441017e0 0x7f5644108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:40.304 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.322+0000 7f5648bff700 1 --2- 172.21.15.17:0/1505630279 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5644100e30 0x7f56441012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:40.304 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.322+0000 7f5648bff700 1 -- 172.21.15.17:0/1505630279 >> 172.21.15.17:0/1505630279 conn(0x7f56440fb1c0 msgr2=0x7f56440fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:40.304 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.322+0000 7f5648bff700 1 -- 172.21.15.17:0/1505630279 shutdown_connections 2023-07-14T11:59:40.304 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.322+0000 7f5648bff700 1 -- 172.21.15.17:0/1505630279 wait complete. 2023-07-14T11:59:40.306 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.323+0000 7f5648bff700 1 Processor -- start 2023-07-14T11:59:40.306 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.323+0000 7f5648bff700 1 -- start start 2023-07-14T11:59:40.306 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.324+0000 7f5648bff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56440ffba0 0x7f5644197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:40.306 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.324+0000 7f5648bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5644100e30 0x7f5644198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:40.307 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.324+0000 7f5648bff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56441017e0 0x7f564419f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:40.307 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.324+0000 7f5648bff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564419f690 con 0x7f56441017e0 2023-07-14T11:59:40.307 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.324+0000 7f5648bff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564419f7d0 con 0x7f56440ffba0 2023-07-14T11:59:40.307 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.324+0000 7f5648bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564419f960 con 0x7f5644100e30 2023-07-14T11:59:40.307 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.324+0000 7f563bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5644100e30 0x7f5644198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:40.307 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.324+0000 7f5642d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56441017e0 0x7f564419f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:40.307 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.324+0000 7f563bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5644100e30 0x7f5644198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.17:49602/0 (socket says 172.21.15.17:49602) 2023-07-14T11:59:40.307 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.324+0000 7f563bfff700 1 -- 172.21.15.17:0/3203870080 learned_addr learned my addr 172.21.15.17:0/3203870080 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:59:40.308 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.324+0000 7f564259c700 1 --2- 172.21.15.17:0/3203870080 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56440ffba0 0x7f5644197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:40.308 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.324+0000 7f563bfff700 1 -- 172.21.15.17:0/3203870080 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56440ffba0 msgr2=0x7f5644197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:40.308 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.324+0000 7f563bfff700 1 --2- 172.21.15.17:0/3203870080 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56440ffba0 0x7f5644197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:40.308 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.324+0000 7f563bfff700 1 -- 172.21.15.17:0/3203870080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56441017e0 msgr2=0x7f564419f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:40.308 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.324+0000 7f563bfff700 1 --2- 172.21.15.17:0/3203870080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56441017e0 0x7f564419f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:40.308 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.324+0000 7f563bfff700 1 -- 172.21.15.17:0/3203870080 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56340095f0 con 0x7f5644100e30 2023-07-14T11:59:40.308 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.324+0000 7f5642d9d700 1 --2- 172.21.15.17:0/3203870080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56441017e0 0x7f564419f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:59:40.309 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.325+0000 7f563bfff700 1 --2- 172.21.15.17:0/3203870080 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5644100e30 0x7f5644198050 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f562c00e9a0 tx=0x7f562c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:40.309 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.325+0000 7f563b7fe700 1 -- 172.21.15.17:0/3203870080 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562c009790 con 0x7f5644100e30 2023-07-14T11:59:40.309 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.325+0000 7f5648bff700 1 -- 172.21.15.17:0/3203870080 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f564419fc40 con 0x7f5644100e30 2023-07-14T11:59:40.309 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.325+0000 7f563b7fe700 1 -- 172.21.15.17:0/3203870080 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f562c010430 con 0x7f5644100e30 2023-07-14T11:59:40.309 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.325+0000 7f563b7fe700 1 -- 172.21.15.17:0/3203870080 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562c018680 con 0x7f5644100e30 2023-07-14T11:59:40.309 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.325+0000 7f5648bff700 1 -- 172.21.15.17:0/3203870080 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56441a01a0 con 0x7f5644100e30 2023-07-14T11:59:40.310 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.326+0000 7f5648bff700 1 -- 172.21.15.17:0/3203870080 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f564410b3c0 con 0x7f5644100e30 2023-07-14T11:59:40.310 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.328+0000 7f563b7fe700 1 -- 172.21.15.17:0/3203870080 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f562c0105a0 con 0x7f5644100e30 2023-07-14T11:59:40.310 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.329+0000 7f563b7fe700 1 --2- 172.21.15.17:0/3203870080 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f56300a1150 0x7f56300a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:40.310 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.330+0000 7f563b7fe700 1 -- 172.21.15.17:0/3203870080 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f562c014070 con 0x7f5644100e30 2023-07-14T11:59:40.311 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.330+0000 7f564259c700 1 --2- 172.21.15.17:0/3203870080 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f56300a1150 0x7f56300a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:40.311 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.330+0000 7f564259c700 1 --2- 172.21.15.17:0/3203870080 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f56300a1150 0x7f56300a3600 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f563400b2e0 tx=0x7f5634005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:40.312 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.331+0000 7f563b7fe700 1 -- 172.21.15.17:0/3203870080 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f562c096190 con 0x7f5644100e30 2023-07-14T11:59:40.539 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.558+0000 7f5648bff700 1 -- 172.21.15.17:0/3203870080 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f56441a0ac0 con 0x7f5644100e30 2023-07-14T11:59:40.539 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.559+0000 7f563b7fe700 1 -- 172.21.15.17:0/3203870080 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v80) v1 ==== 84+0+65 (secure 0 0 0) 0x7f562c0997b0 con 0x7f5644100e30 2023-07-14T11:59:40.540 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:59:40.540 INFO:teuthology.orchestra.run.smithi017.stdout:[ 2023-07-14T11:59:40.540 INFO:teuthology.orchestra.run.smithi017.stdout: { 2023-07-14T11:59:40.540 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 1, 2023-07-14T11:59:40.540 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": ".mgr" 2023-07-14T11:59:40.540 INFO:teuthology.orchestra.run.smithi017.stdout: } 2023-07-14T11:59:40.541 INFO:teuthology.orchestra.run.smithi017.stdout:] 2023-07-14T11:59:40.542 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.561+0000 7f5648bff700 1 -- 172.21.15.17:0/3203870080 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f56300a1150 msgr2=0x7f56300a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:40.542 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.561+0000 7f5648bff700 1 --2- 172.21.15.17:0/3203870080 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f56300a1150 0x7f56300a3600 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f563400b2e0 tx=0x7f5634005fb0 comp rx=0 tx=0).stop 2023-07-14T11:59:40.542 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.561+0000 7f5648bff700 1 -- 172.21.15.17:0/3203870080 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5644100e30 msgr2=0x7f5644198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:40.543 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.561+0000 7f5648bff700 1 --2- 172.21.15.17:0/3203870080 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5644100e30 0x7f5644198050 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f562c00e9a0 tx=0x7f562c00ecb0 comp rx=0 tx=0).stop 2023-07-14T11:59:40.543 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.561+0000 7f5648bff700 1 -- 172.21.15.17:0/3203870080 shutdown_connections 2023-07-14T11:59:40.543 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.561+0000 7f5648bff700 1 --2- 172.21.15.17:0/3203870080 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f56300a1150 0x7f56300a3600 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:40.543 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.561+0000 7f5648bff700 1 --2- 172.21.15.17:0/3203870080 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56440ffba0 0x7f5644197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:40.543 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.561+0000 7f5648bff700 1 --2- 172.21.15.17:0/3203870080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56441017e0 0x7f564419f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:40.544 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.561+0000 7f5648bff700 1 --2- 172.21.15.17:0/3203870080 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5644100e30 0x7f5644198050 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:40.544 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.561+0000 7f5648bff700 1 -- 172.21.15.17:0/3203870080 >> 172.21.15.17:0/3203870080 conn(0x7f56440fb1c0 msgr2=0x7f56440fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:40.544 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.562+0000 7f5648bff700 1 -- 172.21.15.17:0/3203870080 shutdown_connections 2023-07-14T11:59:40.545 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.562+0000 7f5648bff700 1 -- 172.21.15.17:0/3203870080 wait complete. 2023-07-14T11:59:40.563 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-14T11:59:40.563 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-14T11:59:40.782 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.801+0000 7f3ba6fff700 1 -- 172.21.15.17:0/3125064740 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3ba0102f60 msgr2=0x7f3ba0105340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:40.782 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.801+0000 7f3ba6fff700 1 --2- 172.21.15.17:0/3125064740 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3ba0102f60 0x7f3ba0105340 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c009990 tx=0x7f3b8c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:59:40.782 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.801+0000 7f3ba6fff700 1 -- 172.21.15.17:0/3125064740 shutdown_connections 2023-07-14T11:59:40.782 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.801+0000 7f3ba6fff700 1 --2- 172.21.15.17:0/3125064740 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ba0105880 0x7f3ba0107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:40.783 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.801+0000 7f3ba6fff700 1 --2- 172.21.15.17:0/3125064740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ba0100640 0x7f3ba0102a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:40.783 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.801+0000 7f3ba6fff700 1 --2- 172.21.15.17:0/3125064740 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3ba0102f60 0x7f3ba0105340 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:40.783 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.801+0000 7f3ba6fff700 1 -- 172.21.15.17:0/3125064740 >> 172.21.15.17:0/3125064740 conn(0x7f3ba00fa120 msgr2=0x7f3ba00fc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:40.783 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.801+0000 7f3ba6fff700 1 -- 172.21.15.17:0/3125064740 shutdown_connections 2023-07-14T11:59:40.783 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.801+0000 7f3ba6fff700 1 -- 172.21.15.17:0/3125064740 wait complete. 2023-07-14T11:59:40.785 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.802+0000 7f3ba6fff700 1 Processor -- start 2023-07-14T11:59:40.785 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.802+0000 7f3ba6fff700 1 -- start start 2023-07-14T11:59:40.785 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.802+0000 7f3ba6fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ba0100640 0x7f3ba00fffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:40.785 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.803+0000 7f3ba6fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ba0102f60 0x7f3ba00fe5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:40.785 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.803+0000 7f3ba6fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3ba0105880 0x7f3ba00feb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:40.785 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.803+0000 7f3ba6fff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba00ff2d0 con 0x7f3ba0102f60 2023-07-14T11:59:40.786 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.803+0000 7f3ba6fff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba00ff440 con 0x7f3ba0105880 2023-07-14T11:59:40.786 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.803+0000 7f3ba6fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba019f930 con 0x7f3ba0100640 2023-07-14T11:59:40.786 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.803+0000 7f3ba4d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ba0100640 0x7f3ba00fffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:40.786 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.803+0000 7f3b9ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ba0102f60 0x7f3ba00fe5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:40.786 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.803+0000 7f3ba4d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ba0100640 0x7f3ba00fffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.17:49616/0 (socket says 172.21.15.17:49616) 2023-07-14T11:59:40.786 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.803+0000 7f3ba4d9b700 1 -- 172.21.15.17:0/3705733120 learned_addr learned my addr 172.21.15.17:0/3705733120 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:59:40.787 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.803+0000 7f3ba559c700 1 --2- 172.21.15.17:0/3705733120 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3ba0105880 0x7f3ba00feb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:40.787 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.803+0000 7f3ba4d9b700 1 -- 172.21.15.17:0/3705733120 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3ba0105880 msgr2=0x7f3ba00feb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:40.787 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.803+0000 7f3ba4d9b700 1 --2- 172.21.15.17:0/3705733120 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3ba0105880 0x7f3ba00feb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:40.787 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.803+0000 7f3ba4d9b700 1 -- 172.21.15.17:0/3705733120 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ba0102f60 msgr2=0x7f3ba00fe5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:40.787 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.803+0000 7f3ba4d9b700 1 --2- 172.21.15.17:0/3705733120 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ba0102f60 0x7f3ba00fe5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:40.787 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.803+0000 7f3ba4d9b700 1 -- 172.21.15.17:0/3705733120 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b94009520 con 0x7f3ba0100640 2023-07-14T11:59:40.787 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.804+0000 7f3ba4d9b700 1 --2- 172.21.15.17:0/3705733120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ba0100640 0x7f3ba00fffb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3b9400eb30 tx=0x7f3b9400ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:40.788 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.804+0000 7f3b9dffb700 1 -- 172.21.15.17:0/3705733120 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b94009880 con 0x7f3ba0100640 2023-07-14T11:59:40.788 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.804+0000 7f3b9dffb700 1 -- 172.21.15.17:0/3705733120 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b940044e0 con 0x7f3ba0100640 2023-07-14T11:59:40.788 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.804+0000 7f3ba6fff700 1 -- 172.21.15.17:0/3705733120 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b8c0095f0 con 0x7f3ba0100640 2023-07-14T11:59:40.788 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.804+0000 7f3b9dffb700 1 -- 172.21.15.17:0/3705733120 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b94003d80 con 0x7f3ba0100640 2023-07-14T11:59:40.788 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.804+0000 7f3ba6fff700 1 -- 172.21.15.17:0/3705733120 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba019ff90 con 0x7f3ba0100640 2023-07-14T11:59:40.789 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.805+0000 7f3ba6fff700 1 -- 172.21.15.17:0/3705733120 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ba00fcca0 con 0x7f3ba0100640 2023-07-14T11:59:40.789 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.809+0000 7f3b9dffb700 1 -- 172.21.15.17:0/3705733120 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3b94003f20 con 0x7f3ba0100640 2023-07-14T11:59:40.791 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.810+0000 7f3b9dffb700 1 --2- 172.21.15.17:0/3705733120 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3b900a1150 0x7f3b900a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:40.791 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.810+0000 7f3b9dffb700 1 -- 172.21.15.17:0/3705733120 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3b94014070 con 0x7f3ba0100640 2023-07-14T11:59:40.792 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.811+0000 7f3b9ffff700 1 --2- 172.21.15.17:0/3705733120 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3b900a1150 0x7f3b900a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:40.792 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.811+0000 7f3b9dffb700 1 -- 172.21.15.17:0/3705733120 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3b94099d20 con 0x7f3ba0100640 2023-07-14T11:59:40.792 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:40.811+0000 7f3b9ffff700 1 --2- 172.21.15.17:0/3705733120 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3b900a1150 0x7f3b900a3600 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c00b2e0 tx=0x7f3b8c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:40.884 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:40 smithi097 ceph-mon[129313]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:40.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:40 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/3671569470' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:59:40.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:40 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:40.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:40 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:40.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:40 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/3203870080' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-07-14T11:59:40.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:40 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:40.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:40 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:40.894 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:40 smithi017 ceph-mon[126339]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:40.894 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:40 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/3671569470' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:59:40.894 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:40 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:40.894 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:40 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:40.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:40 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/3203870080' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-07-14T11:59:40.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:40 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:40.895 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:40 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:41.023 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.042+0000 7f3ba6fff700 1 -- 172.21.15.17:0/3705733120 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f3ba002c850 con 0x7f3ba0100640 2023-07-14T11:59:41.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:40 smithi093 ceph-mon[128922]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:41.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:40 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/3671569470' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T11:59:41.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:40 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:41.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:40 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:41.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:40 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/3203870080' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-07-14T11:59:41.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:40 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:41.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:40 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:41.715 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.735+0000 7f3b9dffb700 1 -- 172.21.15.17:0/3705733120 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v81) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3b94099fb0 con 0x7f3ba0100640 2023-07-14T11:59:41.720 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.739+0000 7f3ba6fff700 1 -- 172.21.15.17:0/3705733120 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3b900a1150 msgr2=0x7f3b900a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:41.721 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.739+0000 7f3ba6fff700 1 --2- 172.21.15.17:0/3705733120 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3b900a1150 0x7f3b900a3600 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c00b2e0 tx=0x7f3b8c005fb0 comp rx=0 tx=0).stop 2023-07-14T11:59:41.721 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.739+0000 7f3ba6fff700 1 -- 172.21.15.17:0/3705733120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ba0100640 msgr2=0x7f3ba00fffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:41.721 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.739+0000 7f3ba6fff700 1 --2- 172.21.15.17:0/3705733120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ba0100640 0x7f3ba00fffb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3b9400eb30 tx=0x7f3b9400ee40 comp rx=0 tx=0).stop 2023-07-14T11:59:41.721 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.739+0000 7f3ba6fff700 1 -- 172.21.15.17:0/3705733120 shutdown_connections 2023-07-14T11:59:41.721 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.740+0000 7f3ba6fff700 1 --2- 172.21.15.17:0/3705733120 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3b900a1150 0x7f3b900a3600 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:41.722 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.740+0000 7f3ba6fff700 1 --2- 172.21.15.17:0/3705733120 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3ba0105880 0x7f3ba00feb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:41.722 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.740+0000 7f3ba6fff700 1 --2- 172.21.15.17:0/3705733120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ba0100640 0x7f3ba00fffb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:41.722 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.740+0000 7f3ba6fff700 1 --2- 172.21.15.17:0/3705733120 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ba0102f60 0x7f3ba00fe5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:41.722 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.740+0000 7f3ba6fff700 1 -- 172.21.15.17:0/3705733120 >> 172.21.15.17:0/3705733120 conn(0x7f3ba00fa120 msgr2=0x7f3ba0101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:41.722 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.740+0000 7f3ba6fff700 1 -- 172.21.15.17:0/3705733120 shutdown_connections 2023-07-14T11:59:41.722 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.740+0000 7f3ba6fff700 1 -- 172.21.15.17:0/3705733120 wait complete. 2023-07-14T11:59:41.723 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' created 2023-07-14T11:59:41.739 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-07-14T11:59:41.962 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.981+0000 7f0a439ff700 1 -- 172.21.15.17:0/3952390253 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a3c0744b0 msgr2=0x7f0a3c0729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:41.962 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.981+0000 7f0a439ff700 1 --2- 172.21.15.17:0/3952390253 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a3c0744b0 0x7f0a3c0729a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0a38009990 tx=0x7f0a38009ca0 comp rx=0 tx=0).stop 2023-07-14T11:59:41.962 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.981+0000 7f0a439ff700 1 -- 172.21.15.17:0/3952390253 shutdown_connections 2023-07-14T11:59:41.963 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.981+0000 7f0a439ff700 1 --2- 172.21.15.17:0/3952390253 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0a3c107a40 0x7f0a3c109e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:41.963 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.981+0000 7f0a439ff700 1 --2- 172.21.15.17:0/3952390253 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a3c0744b0 0x7f0a3c0729a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:41.963 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.981+0000 7f0a439ff700 1 --2- 172.21.15.17:0/3952390253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a3c072ee0 0x7f0a3c073350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:41.963 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.981+0000 7f0a439ff700 1 -- 172.21.15.17:0/3952390253 >> 172.21.15.17:0/3952390253 conn(0x7f0a3c0fb200 msgr2=0x7f0a3c0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:41.963 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.981+0000 7f0a439ff700 1 -- 172.21.15.17:0/3952390253 shutdown_connections 2023-07-14T11:59:41.963 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.981+0000 7f0a439ff700 1 -- 172.21.15.17:0/3952390253 wait complete. 2023-07-14T11:59:41.965 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.983+0000 7f0a439ff700 1 Processor -- start 2023-07-14T11:59:41.965 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.983+0000 7f0a439ff700 1 -- start start 2023-07-14T11:59:41.965 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.983+0000 7f0a439ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a3c072ee0 0x7f0a3c197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:41.965 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.983+0000 7f0a439ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0a3c0744b0 0x7f0a3c198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:41.965 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.983+0000 7f0a439ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a3c107a40 0x7f0a3c19f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:41.965 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.984+0000 7f0a439ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c19f830 con 0x7f0a3c072ee0 2023-07-14T11:59:41.965 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.984+0000 7f0a439ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c19f9a0 con 0x7f0a3c0744b0 2023-07-14T11:59:41.966 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.984+0000 7f0a439ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c19fb60 con 0x7f0a3c107a40 2023-07-14T11:59:41.966 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.984+0000 7f0a40f9a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0a3c0744b0 0x7f0a3c198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:41.966 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.984+0000 7f0a4179b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a3c072ee0 0x7f0a3c197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:41.966 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.984+0000 7f0a40f9a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0a3c0744b0 0x7f0a3c198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.17:40576/0 (socket says 172.21.15.17:40576) 2023-07-14T11:59:41.966 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.984+0000 7f0a40f9a700 1 -- 172.21.15.17:0/1103229603 learned_addr learned my addr 172.21.15.17:0/1103229603 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:59:41.966 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.984+0000 7f0a41f9c700 1 --2- 172.21.15.17:0/1103229603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a3c107a40 0x7f0a3c19f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:41.966 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.984+0000 7f0a40f9a700 1 -- 172.21.15.17:0/1103229603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a3c107a40 msgr2=0x7f0a3c19f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:41.967 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.984+0000 7f0a40f9a700 1 --2- 172.21.15.17:0/1103229603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a3c107a40 0x7f0a3c19f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:41.967 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.984+0000 7f0a40f9a700 1 -- 172.21.15.17:0/1103229603 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a3c072ee0 msgr2=0x7f0a3c197cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:41.967 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.984+0000 7f0a40f9a700 1 --2- 172.21.15.17:0/1103229603 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a3c072ee0 0x7f0a3c197cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:41.967 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.984+0000 7f0a40f9a700 1 -- 172.21.15.17:0/1103229603 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a380095f0 con 0x7f0a3c0744b0 2023-07-14T11:59:41.968 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.984+0000 7f0a40f9a700 1 --2- 172.21.15.17:0/1103229603 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0a3c0744b0 0x7f0a3c198230 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c00e9a0 tx=0x7f0a2c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:41.968 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.985+0000 7f0a327fc700 1 -- 172.21.15.17:0/1103229603 <== mon.2 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c009790 con 0x7f0a3c0744b0 2023-07-14T11:59:41.968 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.985+0000 7f0a439ff700 1 -- 172.21.15.17:0/1103229603 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a3c19fe40 con 0x7f0a3c0744b0 2023-07-14T11:59:41.968 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.985+0000 7f0a327fc700 1 -- 172.21.15.17:0/1103229603 <== mon.2 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a2c010430 con 0x7f0a3c0744b0 2023-07-14T11:59:41.968 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.985+0000 7f0a327fc700 1 -- 172.21.15.17:0/1103229603 <== mon.2 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c018680 con 0x7f0a3c0744b0 2023-07-14T11:59:41.968 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.985+0000 7f0a439ff700 1 -- 172.21.15.17:0/1103229603 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a3c1a0260 con 0x7f0a3c0744b0 2023-07-14T11:59:41.969 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.986+0000 7f0a439ff700 1 -- 172.21.15.17:0/1103229603 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a3c10d600 con 0x7f0a3c0744b0 2023-07-14T11:59:41.969 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.988+0000 7f0a327fc700 1 -- 172.21.15.17:0/1103229603 <== mon.2 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f0a2c0105a0 con 0x7f0a3c0744b0 2023-07-14T11:59:41.970 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.989+0000 7f0a327fc700 1 --2- 172.21.15.17:0/1103229603 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f0a280a1150 0x7f0a280a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:41.970 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.989+0000 7f0a327fc700 1 -- 172.21.15.17:0/1103229603 <== mon.2 v2:172.21.15.93:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f0a2c014070 con 0x7f0a3c0744b0 2023-07-14T11:59:41.970 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.990+0000 7f0a4179b700 1 --2- 172.21.15.17:0/1103229603 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f0a280a1150 0x7f0a280a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:41.971 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.990+0000 7f0a4179b700 1 --2- 172.21.15.17:0/1103229603 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f0a280a1150 0x7f0a280a3600 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0a38000c00 tx=0x7f0a38005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:41.973 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:41.993+0000 7f0a327fc700 1 -- 172.21.15.17:0/1103229603 <== mon.2 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0a2c051790 con 0x7f0a3c0744b0 2023-07-14T11:59:42.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:41 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:42.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:41 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/3705733120' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-14T11:59:42.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:41 smithi093 ceph-mon[128922]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-14T11:59:42.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:41 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:42.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:41 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:42.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:41 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:59:42.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:41 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:59:42.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:41 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:42.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:41 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:42.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:41 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/3705733120' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-14T11:59:42.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:41 smithi097 ceph-mon[129313]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-14T11:59:42.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:41 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:42.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:41 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:42.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:41 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:59:42.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:41 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:59:42.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:41 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:42.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:41 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:42.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:41 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/3705733120' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-14T11:59:42.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:41 smithi017 ceph-mon[126339]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-14T11:59:42.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:41 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:42.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:41 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:42.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:41 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:59:42.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:41 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:59:42.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:41 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:42.189 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:42.209+0000 7f0a439ff700 1 -- 172.21.15.17:0/1103229603 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f0a3c1a08a0 con 0x7f0a3c0744b0 2023-07-14T11:59:42.723 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:42.742+0000 7f0a327fc700 1 -- 172.21.15.17:0/1103229603 <== mon.2 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v82) v1 ==== 171+0+0 (secure 0 0 0) 0x7f0a2c099cb0 con 0x7f0a3c0744b0 2023-07-14T11:59:42.732 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:42.751+0000 7f0a439ff700 1 -- 172.21.15.17:0/1103229603 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f0a280a1150 msgr2=0x7f0a280a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:42.733 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:42.751+0000 7f0a439ff700 1 --2- 172.21.15.17:0/1103229603 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f0a280a1150 0x7f0a280a3600 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0a38000c00 tx=0x7f0a38005fb0 comp rx=0 tx=0).stop 2023-07-14T11:59:42.733 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:42.751+0000 7f0a439ff700 1 -- 172.21.15.17:0/1103229603 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0a3c0744b0 msgr2=0x7f0a3c198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:42.734 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:42.751+0000 7f0a439ff700 1 --2- 172.21.15.17:0/1103229603 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0a3c0744b0 0x7f0a3c198230 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c00e9a0 tx=0x7f0a2c00ecb0 comp rx=0 tx=0).stop 2023-07-14T11:59:42.734 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:42.752+0000 7f0a439ff700 1 -- 172.21.15.17:0/1103229603 shutdown_connections 2023-07-14T11:59:42.734 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:42.752+0000 7f0a439ff700 1 --2- 172.21.15.17:0/1103229603 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f0a280a1150 0x7f0a280a3600 secure :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0a38000c00 tx=0x7f0a38005fb0 comp rx=0 tx=0).stop 2023-07-14T11:59:42.734 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:42.752+0000 7f0a439ff700 1 --2- 172.21.15.17:0/1103229603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a3c107a40 0x7f0a3c19f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:42.735 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:42.752+0000 7f0a439ff700 1 --2- 172.21.15.17:0/1103229603 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a3c072ee0 0x7f0a3c197cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:42.735 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:42.752+0000 7f0a439ff700 1 --2- 172.21.15.17:0/1103229603 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0a3c0744b0 0x7f0a3c198230 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:42.735 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:42.752+0000 7f0a439ff700 1 -- 172.21.15.17:0/1103229603 >> 172.21.15.17:0/1103229603 conn(0x7f0a3c0fb200 msgr2=0x7f0a3c0fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:42.743 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:42.761+0000 7f0a439ff700 1 -- 172.21.15.17:0/1103229603 shutdown_connections 2023-07-14T11:59:42.744 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:42.762+0000 7f0a439ff700 1 -- 172.21.15.17:0/1103229603 wait complete. 2023-07-14T11:59:42.747 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' created 2023-07-14T11:59:42.767 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-14T11:59:42.981 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.000+0000 7f6439fff700 1 -- 172.21.15.17:0/130378275 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f64340744f0 msgr2=0x7f6434072940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:42.981 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.000+0000 7f6439fff700 1 --2- 172.21.15.17:0/130378275 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f64340744f0 0x7f6434072940 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6424009990 tx=0x7f6424009ca0 comp rx=0 tx=0).stop 2023-07-14T11:59:42.982 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.000+0000 7f6439fff700 1 -- 172.21.15.17:0/130378275 shutdown_connections 2023-07-14T11:59:42.982 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.000+0000 7f6439fff700 1 --2- 172.21.15.17:0/130378275 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f64340744f0 0x7f6434072940 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:42.982 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.000+0000 7f6439fff700 1 --2- 172.21.15.17:0/130378275 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6434107a70 0x7f6434109ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:42.983 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.000+0000 7f6439fff700 1 --2- 172.21.15.17:0/130378275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6434072f10 0x7f6434073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:42.983 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.000+0000 7f6439fff700 1 -- 172.21.15.17:0/130378275 >> 172.21.15.17:0/130378275 conn(0x7f64340fb200 msgr2=0x7f64340fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:42.983 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.000+0000 7f6439fff700 1 -- 172.21.15.17:0/130378275 shutdown_connections 2023-07-14T11:59:42.983 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.000+0000 7f6439fff700 1 -- 172.21.15.17:0/130378275 wait complete. 2023-07-14T11:59:42.985 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.001+0000 7f6439fff700 1 Processor -- start 2023-07-14T11:59:42.985 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.001+0000 7f6439fff700 1 -- start start 2023-07-14T11:59:42.985 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.001+0000 7f6439fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6434072f10 0x7f6434197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:42.985 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.001+0000 7f6439fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64340744f0 0x7f6434197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:42.985 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.001+0000 7f6439fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6434107a70 0x7f643419f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:42.986 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.001+0000 7f6439fff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643419f610 con 0x7f64340744f0 2023-07-14T11:59:42.986 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.001+0000 7f6439fff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643419f750 con 0x7f6434107a70 2023-07-14T11:59:42.986 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.001+0000 7f6439fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643419f8e0 con 0x7f6434072f10 2023-07-14T11:59:42.986 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.002+0000 7f6432ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64340744f0 0x7f6434197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:42.986 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.002+0000 7f64337fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6434072f10 0x7f6434197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:42.986 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.002+0000 7f6433fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6434107a70 0x7f643419f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:42.986 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.002+0000 7f64337fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6434072f10 0x7f6434197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.17:58498/0 (socket says 172.21.15.17:58498) 2023-07-14T11:59:42.986 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.002+0000 7f64337fe700 1 -- 172.21.15.17:0/3577146105 learned_addr learned my addr 172.21.15.17:0/3577146105 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:59:42.987 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.002+0000 7f64337fe700 1 -- 172.21.15.17:0/3577146105 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6434107a70 msgr2=0x7f643419f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:42.987 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.002+0000 7f64337fe700 1 --2- 172.21.15.17:0/3577146105 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6434107a70 0x7f643419f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:42.987 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.002+0000 7f64337fe700 1 -- 172.21.15.17:0/3577146105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64340744f0 msgr2=0x7f6434197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:42.987 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.002+0000 7f64337fe700 1 --2- 172.21.15.17:0/3577146105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64340744f0 0x7f6434197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:42.987 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.002+0000 7f64337fe700 1 -- 172.21.15.17:0/3577146105 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64240095f0 con 0x7f6434072f10 2023-07-14T11:59:42.987 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.002+0000 7f64337fe700 1 --2- 172.21.15.17:0/3577146105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6434072f10 0x7f6434197a90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f64240059d0 tx=0x7f6424004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:42.987 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.002+0000 7f6430ff9700 1 -- 172.21.15.17:0/3577146105 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642401d070 con 0x7f6434072f10 2023-07-14T11:59:42.988 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.003+0000 7f6430ff9700 1 -- 172.21.15.17:0/3577146105 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f642400fb90 con 0x7f6434072f10 2023-07-14T11:59:42.988 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.003+0000 7f6439fff700 1 -- 172.21.15.17:0/3577146105 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f643419fb60 con 0x7f6434072f10 2023-07-14T11:59:42.988 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.003+0000 7f6430ff9700 1 -- 172.21.15.17:0/3577146105 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64240173f0 con 0x7f6434072f10 2023-07-14T11:59:42.988 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.003+0000 7f6439fff700 1 -- 172.21.15.17:0/3577146105 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f643419ffe0 con 0x7f6434072f10 2023-07-14T11:59:42.988 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.004+0000 7f6439fff700 1 -- 172.21.15.17:0/3577146105 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64341061b0 con 0x7f6434072f10 2023-07-14T11:59:42.989 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.006+0000 7f6430ff9700 1 -- 172.21.15.17:0/3577146105 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f642400fd00 con 0x7f6434072f10 2023-07-14T11:59:42.989 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.007+0000 7f6430ff9700 1 --2- 172.21.15.17:0/3577146105 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f641c0a11d0 0x7f641c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:42.989 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.007+0000 7f6430ff9700 1 -- 172.21.15.17:0/3577146105 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f64240cdb40 con 0x7f6434072f10 2023-07-14T11:59:42.989 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.007+0000 7f6432ffd700 1 --2- 172.21.15.17:0/3577146105 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f641c0a11d0 0x7f641c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:42.989 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.008+0000 7f6432ffd700 1 --2- 172.21.15.17:0/3577146105 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f641c0a11d0 0x7f641c0a3680 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6428009c60 tx=0x7f6428009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:42.990 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.009+0000 7f6430ff9700 1 -- 172.21.15.17:0/3577146105 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6424097430 con 0x7f6434072f10 2023-07-14T11:59:43.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:42 smithi093 ceph-mon[128922]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:43.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:42 smithi093 ceph-mon[128922]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-07-14T11:59:43.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:42 smithi093 ceph-mon[128922]: osdmap e81: 12 total, 12 up, 12 in 2023-07-14T11:59:43.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:42 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:59:43.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:42 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/1103229603' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-07-14T11:59:43.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:42 smithi093 ceph-mon[128922]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-07-14T11:59:43.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:42 smithi097 ceph-mon[129313]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:43.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:42 smithi097 ceph-mon[129313]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-07-14T11:59:43.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:42 smithi097 ceph-mon[129313]: osdmap e81: 12 total, 12 up, 12 in 2023-07-14T11:59:43.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:42 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:59:43.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:42 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/1103229603' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-07-14T11:59:43.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:42 smithi097 ceph-mon[129313]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-07-14T11:59:43.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:42 smithi017 ceph-mon[126339]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:43.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:42 smithi017 ceph-mon[126339]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-07-14T11:59:43.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:42 smithi017 ceph-mon[126339]: osdmap e81: 12 total, 12 up, 12 in 2023-07-14T11:59:43.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:42 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T11:59:43.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:42 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/1103229603' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-07-14T11:59:43.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:42 smithi017 ceph-mon[126339]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-07-14T11:59:43.270 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.290+0000 7f6439fff700 1 -- 172.21.15.17:0/3577146105 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f643402c850 con 0x7f6434072f10 2023-07-14T11:59:43.723 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.742+0000 7f6430ff9700 1 -- 172.21.15.17:0/3577146105 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f642409aa50 con 0x7f6434072f10 2023-07-14T11:59:43.727 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.747+0000 7f6439fff700 1 -- 172.21.15.17:0/3577146105 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f641c0a11d0 msgr2=0x7f641c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:43.728 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.747+0000 7f6439fff700 1 --2- 172.21.15.17:0/3577146105 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f641c0a11d0 0x7f641c0a3680 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6428009c60 tx=0x7f6428009260 comp rx=0 tx=0).stop 2023-07-14T11:59:43.728 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.747+0000 7f6439fff700 1 -- 172.21.15.17:0/3577146105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6434072f10 msgr2=0x7f6434197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:43.728 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.747+0000 7f6439fff700 1 --2- 172.21.15.17:0/3577146105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6434072f10 0x7f6434197a90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f64240059d0 tx=0x7f6424004730 comp rx=0 tx=0).stop 2023-07-14T11:59:43.728 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.747+0000 7f6439fff700 1 -- 172.21.15.17:0/3577146105 shutdown_connections 2023-07-14T11:59:43.729 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.747+0000 7f6439fff700 1 --2- 172.21.15.17:0/3577146105 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f641c0a11d0 0x7f641c0a3680 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:43.729 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.747+0000 7f6439fff700 1 --2- 172.21.15.17:0/3577146105 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6434107a70 0x7f643419f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:43.729 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.747+0000 7f6439fff700 1 --2- 172.21.15.17:0/3577146105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6434072f10 0x7f6434197a90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:43.729 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.747+0000 7f6439fff700 1 --2- 172.21.15.17:0/3577146105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64340744f0 0x7f6434197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:43.729 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.747+0000 7f6439fff700 1 -- 172.21.15.17:0/3577146105 >> 172.21.15.17:0/3577146105 conn(0x7f64340fb200 msgr2=0x7f64340fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:43.730 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.748+0000 7f6439fff700 1 -- 172.21.15.17:0/3577146105 shutdown_connections 2023-07-14T11:59:43.730 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.749+0000 7f6439fff700 1 -- 172.21.15.17:0/3577146105 wait complete. 2023-07-14T11:59:43.733 INFO:teuthology.orchestra.run.smithi017.stderr:new fs with metadata pool 2 and data pool 3 2023-07-14T11:59:43.754 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-07-14T11:59:43.755 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-07-14T11:59:43.975 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.994+0000 7f5a377ff700 1 -- 172.21.15.17:0/1651861656 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a30100640 msgr2=0x7f5a30102a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:43.976 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.994+0000 7f5a377ff700 1 --2- 172.21.15.17:0/1651861656 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a30100640 0x7f5a30102a20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c009990 tx=0x7f5a2c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:59:43.976 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.994+0000 7f5a377ff700 1 -- 172.21.15.17:0/1651861656 shutdown_connections 2023-07-14T11:59:43.976 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.994+0000 7f5a377ff700 1 --2- 172.21.15.17:0/1651861656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a30105880 0x7f5a30107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:43.977 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.994+0000 7f5a377ff700 1 --2- 172.21.15.17:0/1651861656 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a30100640 0x7f5a30102a20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:43.977 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.994+0000 7f5a377ff700 1 --2- 172.21.15.17:0/1651861656 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5a30102f60 0x7f5a30105340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:43.978 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.994+0000 7f5a377ff700 1 -- 172.21.15.17:0/1651861656 >> 172.21.15.17:0/1651861656 conn(0x7f5a300fa120 msgr2=0x7f5a300fc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:43.978 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.994+0000 7f5a377ff700 1 -- 172.21.15.17:0/1651861656 shutdown_connections 2023-07-14T11:59:43.978 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.994+0000 7f5a377ff700 1 -- 172.21.15.17:0/1651861656 wait complete. 2023-07-14T11:59:43.982 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.995+0000 7f5a377ff700 1 Processor -- start 2023-07-14T11:59:43.982 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.996+0000 7f5a377ff700 1 -- start start 2023-07-14T11:59:43.982 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.996+0000 7f5a377ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a30100640 0x7f5a30197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:43.983 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.996+0000 7f5a377ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5a30102f60 0x7f5a30198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:43.983 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.996+0000 7f5a377ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a30105880 0x7f5a3019f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:43.983 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.996+0000 7f5a377ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3019f650 con 0x7f5a30100640 2023-07-14T11:59:43.983 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.996+0000 7f5a377ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3019f790 con 0x7f5a30102f60 2023-07-14T11:59:43.984 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.996+0000 7f5a377ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3019f920 con 0x7f5a30105880 2023-07-14T11:59:43.984 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.996+0000 7f5a34d9a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5a30102f60 0x7f5a30198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:43.985 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.996+0000 7f5a3559b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a30100640 0x7f5a30197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:43.985 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.996+0000 7f5a34d9a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5a30102f60 0x7f5a30198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.17:53594/0 (socket says 172.21.15.17:53594) 2023-07-14T11:59:43.985 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.996+0000 7f5a34d9a700 1 -- 172.21.15.17:0/483241015 learned_addr learned my addr 172.21.15.17:0/483241015 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:59:43.985 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.996+0000 7f5a35d9c700 1 --2- 172.21.15.17:0/483241015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a30105880 0x7f5a3019f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:43.985 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.997+0000 7f5a3559b700 1 -- 172.21.15.17:0/483241015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a30105880 msgr2=0x7f5a3019f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:43.986 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.997+0000 7f5a3559b700 1 --2- 172.21.15.17:0/483241015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a30105880 0x7f5a3019f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:43.986 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.997+0000 7f5a3559b700 1 -- 172.21.15.17:0/483241015 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5a30102f60 msgr2=0x7f5a30198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:43.986 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.997+0000 7f5a3559b700 1 --2- 172.21.15.17:0/483241015 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5a30102f60 0x7f5a30198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:43.986 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.997+0000 7f5a3559b700 1 -- 172.21.15.17:0/483241015 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a2c0095f0 con 0x7f5a30100640 2023-07-14T11:59:43.986 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.997+0000 7f5a3559b700 1 --2- 172.21.15.17:0/483241015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a30100640 0x7f5a30197ad0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c00b680 tx=0x7f5a2c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:43.986 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.997+0000 7f5a267fc700 1 -- 172.21.15.17:0/483241015 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c01d070 con 0x7f5a30100640 2023-07-14T11:59:43.987 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.997+0000 7f5a267fc700 1 -- 172.21.15.17:0/483241015 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a2c021410 con 0x7f5a30100640 2023-07-14T11:59:43.987 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.997+0000 7f5a377ff700 1 -- 172.21.15.17:0/483241015 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a3019fba0 con 0x7f5a30100640 2023-07-14T11:59:43.987 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.997+0000 7f5a267fc700 1 -- 172.21.15.17:0/483241015 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c021ea0 con 0x7f5a30100640 2023-07-14T11:59:43.987 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.998+0000 7f5a377ff700 1 -- 172.21.15.17:0/483241015 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a301a0020 con 0x7f5a30100640 2023-07-14T11:59:43.987 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:43.999+0000 7f5a377ff700 1 -- 172.21.15.17:0/483241015 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a3010b3c0 con 0x7f5a30100640 2023-07-14T11:59:43.987 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.001+0000 7f5a267fc700 1 -- 172.21.15.17:0/483241015 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5a2c0217f0 con 0x7f5a30100640 2023-07-14T11:59:43.988 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.002+0000 7f5a267fc700 1 --2- 172.21.15.17:0/483241015 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f5a1c0a11d0 0x7f5a1c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:43.988 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.002+0000 7f5a267fc700 1 -- 172.21.15.17:0/483241015 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5a2c0cd920 con 0x7f5a30100640 2023-07-14T11:59:43.988 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.003+0000 7f5a34d9a700 1 --2- 172.21.15.17:0/483241015 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f5a1c0a11d0 0x7f5a1c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:43.988 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.003+0000 7f5a34d9a700 1 --2- 172.21.15.17:0/483241015 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f5a1c0a11d0 0x7f5a1c0a3680 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5a20009d20 tx=0x7f5a20009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:43.990 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.009+0000 7f5a267fc700 1 -- 172.21.15.17:0/483241015 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5a2c0972d0 con 0x7f5a30100640 2023-07-14T11:59:44.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:43 smithi093 ceph-mon[128922]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-07-14T11:59:44.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:43 smithi093 ceph-mon[128922]: osdmap e82: 12 total, 12 up, 12 in 2023-07-14T11:59:44.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:43 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/3577146105' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-14T11:59:44.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:43 smithi093 ceph-mon[128922]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-14T11:59:44.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:43 smithi093 ceph-mon[128922]: daemon mds.a assigned to filesystem cephfs as rank 0 2023-07-14T11:59:44.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:43 smithi093 ceph-mon[128922]: osdmap e83: 12 total, 12 up, 12 in 2023-07-14T11:59:44.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:43 smithi097 ceph-mon[129313]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-07-14T11:59:44.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:43 smithi097 ceph-mon[129313]: osdmap e82: 12 total, 12 up, 12 in 2023-07-14T11:59:44.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:43 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/3577146105' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-14T11:59:44.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:43 smithi097 ceph-mon[129313]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-14T11:59:44.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:43 smithi097 ceph-mon[129313]: daemon mds.a assigned to filesystem cephfs as rank 0 2023-07-14T11:59:44.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:43 smithi097 ceph-mon[129313]: osdmap e83: 12 total, 12 up, 12 in 2023-07-14T11:59:44.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:43 smithi017 ceph-mon[126339]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-07-14T11:59:44.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:43 smithi017 ceph-mon[126339]: osdmap e82: 12 total, 12 up, 12 in 2023-07-14T11:59:44.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:43 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/3577146105' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-14T11:59:44.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:43 smithi017 ceph-mon[126339]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-14T11:59:44.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:43 smithi017 ceph-mon[126339]: daemon mds.a assigned to filesystem cephfs as rank 0 2023-07-14T11:59:44.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:43 smithi017 ceph-mon[126339]: osdmap e83: 12 total, 12 up, 12 in 2023-07-14T11:59:44.245 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.264+0000 7f5a377ff700 1 -- 172.21.15.17:0/483241015 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f5a3010b580 con 0x7f5a30100640 2023-07-14T11:59:44.432 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.451+0000 7f5a267fc700 1 -- 172.21.15.17:0/483241015 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v84) v1 ==== 157+0+0 (secure 0 0 0) 0x7f5a2c026020 con 0x7f5a30100640 2023-07-14T11:59:44.435 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.454+0000 7f5a377ff700 1 -- 172.21.15.17:0/483241015 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f5a1c0a11d0 msgr2=0x7f5a1c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:44.435 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.454+0000 7f5a377ff700 1 --2- 172.21.15.17:0/483241015 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f5a1c0a11d0 0x7f5a1c0a3680 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5a20009d20 tx=0x7f5a20009260 comp rx=0 tx=0).stop 2023-07-14T11:59:44.435 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.454+0000 7f5a377ff700 1 -- 172.21.15.17:0/483241015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a30100640 msgr2=0x7f5a30197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:44.435 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.454+0000 7f5a377ff700 1 --2- 172.21.15.17:0/483241015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a30100640 0x7f5a30197ad0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c00b680 tx=0x7f5a2c005740 comp rx=0 tx=0).stop 2023-07-14T11:59:44.435 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.454+0000 7f5a377ff700 1 -- 172.21.15.17:0/483241015 shutdown_connections 2023-07-14T11:59:44.436 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.454+0000 7f5a377ff700 1 --2- 172.21.15.17:0/483241015 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f5a1c0a11d0 0x7f5a1c0a3680 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:44.436 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.454+0000 7f5a377ff700 1 --2- 172.21.15.17:0/483241015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a30105880 0x7f5a3019f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:44.436 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.454+0000 7f5a377ff700 1 --2- 172.21.15.17:0/483241015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a30100640 0x7f5a30197ad0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:44.436 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.454+0000 7f5a377ff700 1 --2- 172.21.15.17:0/483241015 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5a30102f60 0x7f5a30198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:44.436 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.454+0000 7f5a377ff700 1 -- 172.21.15.17:0/483241015 >> 172.21.15.17:0/483241015 conn(0x7f5a300fa120 msgr2=0x7f5a30101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:44.436 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.454+0000 7f5a377ff700 1 -- 172.21.15.17:0/483241015 shutdown_connections 2023-07-14T11:59:44.436 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.454+0000 7f5a377ff700 1 -- 172.21.15.17:0/483241015 wait complete. 2023-07-14T11:59:44.458 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-07-14T11:59:44.682 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.701+0000 7fd42b59e700 1 -- 172.21.15.17:0/3844548606 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd42c068500 msgr2=0x7fd42c1024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:44.682 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.701+0000 7fd42b59e700 1 --2- 172.21.15.17:0/3844548606 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd42c068500 0x7fd42c1024c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd41c009990 tx=0x7fd41c009ca0 comp rx=0 tx=0).stop 2023-07-14T11:59:44.682 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.701+0000 7fd42b59e700 1 -- 172.21.15.17:0/3844548606 shutdown_connections 2023-07-14T11:59:44.682 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.701+0000 7fd42b59e700 1 --2- 172.21.15.17:0/3844548606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd42c1054f0 0x7fd42c1079a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:44.683 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.701+0000 7fd42b59e700 1 --2- 172.21.15.17:0/3844548606 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd42c068500 0x7fd42c1024c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:44.683 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.701+0000 7fd42b59e700 1 --2- 172.21.15.17:0/3844548606 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd42c102a00 0x7fd42c104e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:44.683 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.701+0000 7fd42b59e700 1 -- 172.21.15.17:0/3844548606 >> 172.21.15.17:0/3844548606 conn(0x7fd42c0f9e50 msgr2=0x7fd42c0fc260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:44.683 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.701+0000 7fd42b59e700 1 -- 172.21.15.17:0/3844548606 shutdown_connections 2023-07-14T11:59:44.683 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.701+0000 7fd42b59e700 1 -- 172.21.15.17:0/3844548606 wait complete. 2023-07-14T11:59:44.685 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.702+0000 7fd42b59e700 1 Processor -- start 2023-07-14T11:59:44.685 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.702+0000 7fd42b59e700 1 -- start start 2023-07-14T11:59:44.685 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.702+0000 7fd42b59e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd42c068500 0x7fd42c1978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:44.685 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.702+0000 7fd42b59e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd42c102a00 0x7fd42c197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:44.685 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.702+0000 7fd42b59e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd42c1054f0 0x7fd42c19eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:44.686 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.702+0000 7fd42b59e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42c19f470 con 0x7fd42c1054f0 2023-07-14T11:59:44.686 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.702+0000 7fd42b59e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42c19f5b0 con 0x7fd42c102a00 2023-07-14T11:59:44.686 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.702+0000 7fd42b59e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42c19f740 con 0x7fd42c068500 2023-07-14T11:59:44.686 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.703+0000 7fd423fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd42c102a00 0x7fd42c197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:44.686 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.703+0000 7fd423fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd42c102a00 0x7fd42c197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.17:53608/0 (socket says 172.21.15.17:53608) 2023-07-14T11:59:44.686 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.703+0000 7fd423fff700 1 -- 172.21.15.17:0/1711578878 learned_addr learned my addr 172.21.15.17:0/1711578878 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:59:44.686 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.703+0000 7fd42ad9d700 1 --2- 172.21.15.17:0/1711578878 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd42c1054f0 0x7fd42c19eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:44.686 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.703+0000 7fd42a59c700 1 --2- 172.21.15.17:0/1711578878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd42c068500 0x7fd42c1978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:44.687 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.703+0000 7fd423fff700 1 -- 172.21.15.17:0/1711578878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd42c068500 msgr2=0x7fd42c1978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:44.687 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.703+0000 7fd423fff700 1 --2- 172.21.15.17:0/1711578878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd42c068500 0x7fd42c1978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:44.687 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.703+0000 7fd423fff700 1 -- 172.21.15.17:0/1711578878 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd42c1054f0 msgr2=0x7fd42c19eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:44.687 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.703+0000 7fd423fff700 1 --2- 172.21.15.17:0/1711578878 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd42c1054f0 0x7fd42c19eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:44.687 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.703+0000 7fd423fff700 1 -- 172.21.15.17:0/1711578878 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd41c0095f0 con 0x7fd42c102a00 2023-07-14T11:59:44.687 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.703+0000 7fd42ad9d700 1 --2- 172.21.15.17:0/1711578878 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd42c1054f0 0x7fd42c19eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:59:44.687 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.703+0000 7fd42a59c700 1 --2- 172.21.15.17:0/1711578878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd42c068500 0x7fd42c1978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:59:44.688 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.703+0000 7fd423fff700 1 --2- 172.21.15.17:0/1711578878 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd42c102a00 0x7fd42c197e30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd41400e9a0 tx=0x7fd41400ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:44.688 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.703+0000 7fd4237fe700 1 -- 172.21.15.17:0/1711578878 <== mon.2 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd414009790 con 0x7fd42c102a00 2023-07-14T11:59:44.688 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.703+0000 7fd4237fe700 1 -- 172.21.15.17:0/1711578878 <== mon.2 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd41400cbd0 con 0x7fd42c102a00 2023-07-14T11:59:44.688 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.703+0000 7fd42b59e700 1 -- 172.21.15.17:0/1711578878 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd42c19fa20 con 0x7fd42c102a00 2023-07-14T11:59:44.688 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.703+0000 7fd4237fe700 1 -- 172.21.15.17:0/1711578878 <== mon.2 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd414018680 con 0x7fd42c102a00 2023-07-14T11:59:44.688 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.704+0000 7fd42b59e700 1 -- 172.21.15.17:0/1711578878 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd42c19ff80 con 0x7fd42c102a00 2023-07-14T11:59:44.689 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.704+0000 7fd42b59e700 1 -- 172.21.15.17:0/1711578878 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd42c10b130 con 0x7fd42c102a00 2023-07-14T11:59:44.689 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.706+0000 7fd4237fe700 1 -- 172.21.15.17:0/1711578878 <== mon.2 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd41400cd40 con 0x7fd42c102a00 2023-07-14T11:59:44.689 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.708+0000 7fd4237fe700 1 --2- 172.21.15.17:0/1711578878 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fd4180a11d0 0x7fd4180a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:44.690 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.708+0000 7fd4237fe700 1 -- 172.21.15.17:0/1711578878 <== mon.2 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8205+0+0 (secure 0 0 0) 0x7fd414014070 con 0x7fd42c102a00 2023-07-14T11:59:44.690 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.708+0000 7fd42a59c700 1 --2- 172.21.15.17:0/1711578878 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fd4180a11d0 0x7fd4180a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:44.690 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.708+0000 7fd42a59c700 1 --2- 172.21.15.17:0/1711578878 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fd4180a11d0 0x7fd4180a3680 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd41c00b2e0 tx=0x7fd41c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:44.690 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.710+0000 7fd4237fe700 1 -- 172.21.15.17:0/1711578878 <== mon.2 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd414096370 con 0x7fd42c102a00 2023-07-14T11:59:44.906 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:44.926+0000 7fd42b59e700 1 -- 172.21.15.17:0/1711578878 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd42c10b2c0 con 0x7fd42c102a00 2023-07-14T11:59:45.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:44 smithi093 ceph-mon[128922]: pgmap v272: 97 pgs: 25 creating+peering, 21 active+clean, 51 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:45.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:44 smithi093 ceph-mon[128922]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-07-14T11:59:45.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:44 smithi093 ceph-mon[128922]: fsmap cephfs:1 {0=a=up:creating} 11 up:standby 2023-07-14T11:59:45.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:44 smithi093 ceph-mon[128922]: daemon mds.a is now active in filesystem cephfs as rank 0 2023-07-14T11:59:45.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:44 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/483241015' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-07-14T11:59:45.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:44 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/483241015' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-07-14T11:59:45.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:44 smithi093 ceph-mon[128922]: osdmap e84: 12 total, 12 up, 12 in 2023-07-14T11:59:45.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:44 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:45.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:44 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:45.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:44 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:45.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:44 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:45.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:44 smithi097 ceph-mon[129313]: pgmap v272: 97 pgs: 25 creating+peering, 21 active+clean, 51 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:45.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:44 smithi097 ceph-mon[129313]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-07-14T11:59:45.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:44 smithi097 ceph-mon[129313]: fsmap cephfs:1 {0=a=up:creating} 11 up:standby 2023-07-14T11:59:45.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:44 smithi097 ceph-mon[129313]: daemon mds.a is now active in filesystem cephfs as rank 0 2023-07-14T11:59:45.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:44 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/483241015' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-07-14T11:59:45.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:44 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/483241015' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-07-14T11:59:45.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:44 smithi097 ceph-mon[129313]: osdmap e84: 12 total, 12 up, 12 in 2023-07-14T11:59:45.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:44 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:45.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:44 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:45.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:44 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:45.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:44 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:45.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:44 smithi017 ceph-mon[126339]: pgmap v272: 97 pgs: 25 creating+peering, 21 active+clean, 51 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T11:59:45.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:44 smithi017 ceph-mon[126339]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-07-14T11:59:45.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:44 smithi017 ceph-mon[126339]: fsmap cephfs:1 {0=a=up:creating} 11 up:standby 2023-07-14T11:59:45.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:44 smithi017 ceph-mon[126339]: daemon mds.a is now active in filesystem cephfs as rank 0 2023-07-14T11:59:45.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:44 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/483241015' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-07-14T11:59:45.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:44 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/483241015' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-07-14T11:59:45.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:44 smithi017 ceph-mon[126339]: osdmap e84: 12 total, 12 up, 12 in 2023-07-14T11:59:45.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:44 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:45.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:44 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:45.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:44 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:45.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:44 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:45.983 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:45 smithi017 ceph-mon[126339]: mds.? [v2:172.21.15.17:6840/3905201143,v1:172.21.15.17:6841/3905201143] up:active 2023-07-14T11:59:45.984 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:45 smithi017 ceph-mon[126339]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2023-07-14T11:59:45.984 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:45 smithi017 ceph-mon[126339]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2023-07-14T11:59:45.984 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:45 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/1711578878' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-14T11:59:45.984 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:45 smithi017 ceph-mon[126339]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-14T11:59:45.984 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:45 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:45.985 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:45 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:45.985 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:45 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:59:45.985 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:45 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:59:45.985 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:45 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:46.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:45 smithi093 ceph-mon[128922]: mds.? [v2:172.21.15.17:6840/3905201143,v1:172.21.15.17:6841/3905201143] up:active 2023-07-14T11:59:46.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:45 smithi093 ceph-mon[128922]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2023-07-14T11:59:46.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:45 smithi093 ceph-mon[128922]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2023-07-14T11:59:46.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:45 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/1711578878' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-14T11:59:46.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:45 smithi093 ceph-mon[128922]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-14T11:59:46.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:45 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:46.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:45 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:46.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:45 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:59:46.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:45 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:59:46.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:45 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:46.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:45 smithi097 ceph-mon[129313]: mds.? [v2:172.21.15.17:6840/3905201143,v1:172.21.15.17:6841/3905201143] up:active 2023-07-14T11:59:46.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:45 smithi097 ceph-mon[129313]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2023-07-14T11:59:46.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:45 smithi097 ceph-mon[129313]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2023-07-14T11:59:46.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:45 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/1711578878' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-14T11:59:46.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:45 smithi097 ceph-mon[129313]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-14T11:59:46.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:45 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:46.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:45 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:46.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:45 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T11:59:46.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:45 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T11:59:46.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:45 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T11:59:46.394 INFO:journalctl@ceph.mds.a.smithi017.stdout:Jul 14 11:59:46 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-a[161183]: 2023-07-14T11:59:46.001+0000 7fcd1b40b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-14T11:59:46.747 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:46.766+0000 7fd4237fe700 1 -- 172.21.15.17:0/1711578878 <== mon.2 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v86) v1 ==== 228+0+0 (secure 0 0 0) 0x7fd414099990 con 0x7fd42c102a00 2023-07-14T11:59:46.753 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:46.772+0000 7fd42b59e700 1 -- 172.21.15.17:0/1711578878 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fd4180a11d0 msgr2=0x7fd4180a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:46.754 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:46.772+0000 7fd42b59e700 1 --2- 172.21.15.17:0/1711578878 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fd4180a11d0 0x7fd4180a3680 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd41c00b2e0 tx=0x7fd41c005fd0 comp rx=0 tx=0).stop 2023-07-14T11:59:46.754 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:46.772+0000 7fd42b59e700 1 -- 172.21.15.17:0/1711578878 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd42c102a00 msgr2=0x7fd42c197e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:46.754 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:46.772+0000 7fd42b59e700 1 --2- 172.21.15.17:0/1711578878 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd42c102a00 0x7fd42c197e30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd41400e9a0 tx=0x7fd41400ed60 comp rx=0 tx=0).stop 2023-07-14T11:59:46.754 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:46.773+0000 7fd42b59e700 1 -- 172.21.15.17:0/1711578878 shutdown_connections 2023-07-14T11:59:46.754 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:46.773+0000 7fd42b59e700 1 --2- 172.21.15.17:0/1711578878 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fd4180a11d0 0x7fd4180a3680 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:46.754 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:46.773+0000 7fd42b59e700 1 --2- 172.21.15.17:0/1711578878 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd42c1054f0 0x7fd42c19eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:46.754 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:46.773+0000 7fd42b59e700 1 --2- 172.21.15.17:0/1711578878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd42c068500 0x7fd42c1978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:46.755 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:46.773+0000 7fd42b59e700 1 --2- 172.21.15.17:0/1711578878 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd42c102a00 0x7fd42c197e30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:46.755 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:46.773+0000 7fd42b59e700 1 -- 172.21.15.17:0/1711578878 >> 172.21.15.17:0/1711578878 conn(0x7fd42c0f9e50 msgr2=0x7fd42c0fc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:46.755 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:46.773+0000 7fd42b59e700 1 -- 172.21.15.17:0/1711578878 shutdown_connections 2023-07-14T11:59:46.755 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:46.773+0000 7fd42b59e700 1 -- 172.21.15.17:0/1711578878 wait complete. 2023-07-14T11:59:46.755 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data_ec' created 2023-07-14T11:59:46.774 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-07-14T11:59:46.986 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.005+0000 7fc2e2bff700 1 -- 172.21.15.17:0/300370970 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc2dc102f60 msgr2=0x7fc2dc105340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:46.987 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.005+0000 7fc2e2bff700 1 --2- 172.21.15.17:0/300370970 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc2dc102f60 0x7fc2dc105340 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc2c80098f0 tx=0x7fc2c8009c00 comp rx=0 tx=0).stop 2023-07-14T11:59:46.987 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.005+0000 7fc2e2bff700 1 -- 172.21.15.17:0/300370970 shutdown_connections 2023-07-14T11:59:46.988 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.005+0000 7fc2e2bff700 1 --2- 172.21.15.17:0/300370970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2dc105880 0x7fc2dc107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:46.988 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.005+0000 7fc2e2bff700 1 --2- 172.21.15.17:0/300370970 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc2dc100640 0x7fc2dc102a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:46.988 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.005+0000 7fc2e2bff700 1 --2- 172.21.15.17:0/300370970 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc2dc102f60 0x7fc2dc105340 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:46.988 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.005+0000 7fc2e2bff700 1 -- 172.21.15.17:0/300370970 >> 172.21.15.17:0/300370970 conn(0x7fc2dc0fa120 msgr2=0x7fc2dc0fc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:46.989 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.005+0000 7fc2e2bff700 1 -- 172.21.15.17:0/300370970 shutdown_connections 2023-07-14T11:59:46.989 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.005+0000 7fc2e2bff700 1 -- 172.21.15.17:0/300370970 wait complete. 2023-07-14T11:59:46.989 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.006+0000 7fc2e2bff700 1 Processor -- start 2023-07-14T11:59:46.990 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.006+0000 7fc2e2bff700 1 -- start start 2023-07-14T11:59:46.993 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.006+0000 7fc2e2bff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc2dc100640 0x7fc2dc197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:46.994 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.006+0000 7fc2e2bff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc2dc102f60 0x7fc2dc198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:46.994 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.006+0000 7fc2e2bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2dc105880 0x7fc2dc19f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:46.994 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.006+0000 7fc2e2bff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2dc19f6b0 con 0x7fc2dc102f60 2023-07-14T11:59:46.994 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.006+0000 7fc2e2bff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2dc19f7f0 con 0x7fc2dc100640 2023-07-14T11:59:46.995 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.006+0000 7fc2e2bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2dc19f980 con 0x7fc2dc105880 2023-07-14T11:59:46.995 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.007+0000 7fc2e099b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc2dc100640 0x7fc2dc197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:46.995 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.007+0000 7fc2dbfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc2dc102f60 0x7fc2dc198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:46.995 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.007+0000 7fc2e099b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc2dc100640 0x7fc2dc197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.17:53628/0 (socket says 172.21.15.17:53628) 2023-07-14T11:59:46.996 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.007+0000 7fc2e099b700 1 -- 172.21.15.17:0/1593466105 learned_addr learned my addr 172.21.15.17:0/1593466105 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:59:46.996 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.007+0000 7fc2e119c700 1 --2- 172.21.15.17:0/1593466105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2dc105880 0x7fc2dc19f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:46.996 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.007+0000 7fc2e099b700 1 -- 172.21.15.17:0/1593466105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2dc105880 msgr2=0x7fc2dc19f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:46.997 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.007+0000 7fc2e099b700 1 --2- 172.21.15.17:0/1593466105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2dc105880 0x7fc2dc19f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:46.997 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.007+0000 7fc2e099b700 1 -- 172.21.15.17:0/1593466105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc2dc102f60 msgr2=0x7fc2dc198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:46.997 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.007+0000 7fc2e099b700 1 --2- 172.21.15.17:0/1593466105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc2dc102f60 0x7fc2dc198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:46.997 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.007+0000 7fc2e099b700 1 -- 172.21.15.17:0/1593466105 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d00095f0 con 0x7fc2dc100640 2023-07-14T11:59:46.997 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.007+0000 7fc2e099b700 1 --2- 172.21.15.17:0/1593466105 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc2dc100640 0x7fc2dc197b30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc2d000ec10 tx=0x7fc2d000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:46.997 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.008+0000 7fc2d9ffb700 1 -- 172.21.15.17:0/1593466105 <== mon.2 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d000cc90 con 0x7fc2dc100640 2023-07-14T11:59:46.997 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.008+0000 7fc2e2bff700 1 -- 172.21.15.17:0/1593466105 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2c8009520 con 0x7fc2dc100640 2023-07-14T11:59:46.998 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.008+0000 7fc2d9ffb700 1 -- 172.21.15.17:0/1593466105 <== mon.2 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2d00044e0 con 0x7fc2dc100640 2023-07-14T11:59:46.998 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.008+0000 7fc2d9ffb700 1 -- 172.21.15.17:0/1593466105 <== mon.2 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d0010430 con 0x7fc2dc100640 2023-07-14T11:59:46.998 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.008+0000 7fc2e2bff700 1 -- 172.21.15.17:0/1593466105 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2dc1a0030 con 0x7fc2dc100640 2023-07-14T11:59:46.998 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.009+0000 7fc2e2bff700 1 -- 172.21.15.17:0/1593466105 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2dc10b3d0 con 0x7fc2dc100640 2023-07-14T11:59:46.998 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.011+0000 7fc2d9ffb700 1 -- 172.21.15.17:0/1593466105 <== mon.2 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc2d0003680 con 0x7fc2dc100640 2023-07-14T11:59:46.998 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.012+0000 7fc2d9ffb700 1 --2- 172.21.15.17:0/1593466105 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fc2cc0a11d0 0x7fc2cc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:46.998 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.012+0000 7fc2d9ffb700 1 -- 172.21.15.17:0/1593466105 <== mon.2 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8698+0+0 (secure 0 0 0) 0x7fc2d0014070 con 0x7fc2dc100640 2023-07-14T11:59:46.999 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.012+0000 7fc2dbfff700 1 --2- 172.21.15.17:0/1593466105 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fc2cc0a11d0 0x7fc2cc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:46.999 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.013+0000 7fc2dbfff700 1 --2- 172.21.15.17:0/1593466105 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fc2cc0a11d0 0x7fc2cc0a3680 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8005e90 tx=0x7fc2c8019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:46.999 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.014+0000 7fc2d9ffb700 1 -- 172.21.15.17:0/1593466105 <== mon.2 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc2d0096ea0 con 0x7fc2dc100640 2023-07-14T11:59:47.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:46 smithi093 ceph-mon[128922]: pgmap v274: 97 pgs: 25 creating+peering, 36 active+clean, 36 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2023-07-14T11:59:47.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:46 smithi093 ceph-mon[128922]: osdmap e85: 12 total, 12 up, 12 in 2023-07-14T11:59:47.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:46 smithi097 ceph-mon[129313]: pgmap v274: 97 pgs: 25 creating+peering, 36 active+clean, 36 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2023-07-14T11:59:47.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:46 smithi097 ceph-mon[129313]: osdmap e85: 12 total, 12 up, 12 in 2023-07-14T11:59:47.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:46 smithi017 ceph-mon[126339]: pgmap v274: 97 pgs: 25 creating+peering, 36 active+clean, 36 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2023-07-14T11:59:47.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:46 smithi017 ceph-mon[126339]: osdmap e85: 12 total, 12 up, 12 in 2023-07-14T11:59:47.201 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.220+0000 7fc2e2bff700 1 -- 172.21.15.17:0/1593466105 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fc2dc10b590 con 0x7fc2dc100640 2023-07-14T11:59:47.747 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.767+0000 7fc2d9ffb700 1 -- 172.21.15.17:0/1593466105 <== mon.2 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v87) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc2d009a4c0 con 0x7fc2dc100640 2023-07-14T11:59:47.761 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.780+0000 7fc2e2bff700 1 -- 172.21.15.17:0/1593466105 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fc2cc0a11d0 msgr2=0x7fc2cc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:47.761 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.780+0000 7fc2e2bff700 1 --2- 172.21.15.17:0/1593466105 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fc2cc0a11d0 0x7fc2cc0a3680 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8005e90 tx=0x7fc2c8019040 comp rx=0 tx=0).stop 2023-07-14T11:59:47.761 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.780+0000 7fc2e2bff700 1 -- 172.21.15.17:0/1593466105 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc2dc100640 msgr2=0x7fc2dc197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:47.761 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.780+0000 7fc2e2bff700 1 --2- 172.21.15.17:0/1593466105 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc2dc100640 0x7fc2dc197b30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc2d000ec10 tx=0x7fc2d000ef20 comp rx=0 tx=0).stop 2023-07-14T11:59:47.765 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.784+0000 7fc2e2bff700 1 -- 172.21.15.17:0/1593466105 shutdown_connections 2023-07-14T11:59:47.765 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.784+0000 7fc2e2bff700 1 --2- 172.21.15.17:0/1593466105 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fc2cc0a11d0 0x7fc2cc0a3680 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:47.765 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.784+0000 7fc2e2bff700 1 --2- 172.21.15.17:0/1593466105 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc2dc100640 0x7fc2dc197b30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:47.765 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.784+0000 7fc2e2bff700 1 --2- 172.21.15.17:0/1593466105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2dc105880 0x7fc2dc19f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:47.765 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.784+0000 7fc2e2bff700 1 --2- 172.21.15.17:0/1593466105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc2dc102f60 0x7fc2dc198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:47.765 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.784+0000 7fc2e2bff700 1 -- 172.21.15.17:0/1593466105 >> 172.21.15.17:0/1593466105 conn(0x7fc2dc0fa120 msgr2=0x7fc2dc101ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:47.766 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.785+0000 7fc2e2bff700 1 -- 172.21.15.17:0/1593466105 shutdown_connections 2023-07-14T11:59:47.766 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:47.785+0000 7fc2e2bff700 1 -- 172.21.15.17:0/1593466105 wait complete. 2023-07-14T11:59:47.769 INFO:teuthology.orchestra.run.smithi017.stderr:set pool 4 allow_ec_overwrites to true 2023-07-14T11:59:47.793 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-07-14T11:59:48.005 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.024+0000 7fce01dff700 1 -- 172.21.15.17:0/684577671 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdfc0ffbe0 msgr2=0x7fcdfc0ffff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:48.005 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.024+0000 7fce01dff700 1 --2- 172.21.15.17:0/684577671 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdfc0ffbe0 0x7fcdfc0ffff0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcdec009990 tx=0x7fcdec009ca0 comp rx=0 tx=0).stop 2023-07-14T11:59:48.005 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.024+0000 7fce01dff700 1 -- 172.21.15.17:0/684577671 shutdown_connections 2023-07-14T11:59:48.005 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.024+0000 7fce01dff700 1 --2- 172.21.15.17:0/684577671 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcdfc101760 0x7fcdfc108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:48.006 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.024+0000 7fce01dff700 1 --2- 172.21.15.17:0/684577671 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdfc0ffbe0 0x7fcdfc0ffff0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:48.006 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.024+0000 7fce01dff700 1 --2- 172.21.15.17:0/684577671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdfc100dd0 0x7fcdfc101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:48.006 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.024+0000 7fce01dff700 1 -- 172.21.15.17:0/684577671 >> 172.21.15.17:0/684577671 conn(0x7fcdfc0fb200 msgr2=0x7fcdfc0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:48.006 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.024+0000 7fce01dff700 1 -- 172.21.15.17:0/684577671 shutdown_connections 2023-07-14T11:59:48.006 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.024+0000 7fce01dff700 1 -- 172.21.15.17:0/684577671 wait complete. 2023-07-14T11:59:48.008 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.025+0000 7fce01dff700 1 Processor -- start 2023-07-14T11:59:48.008 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.025+0000 7fce01dff700 1 -- start start 2023-07-14T11:59:48.008 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.025+0000 7fce01dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdfc0ffbe0 0x7fcdfc197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:48.008 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.025+0000 7fce01dff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcdfc100dd0 0x7fcdfc1980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:48.009 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.025+0000 7fce01dff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdfc101760 0x7fcdfc19f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:48.009 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.025+0000 7fce01dff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdfc19f730 con 0x7fcdfc101760 2023-07-14T11:59:48.009 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.025+0000 7fce01dff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdfc19f870 con 0x7fcdfc100dd0 2023-07-14T11:59:48.009 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.025+0000 7fce01dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdfc19fa00 con 0x7fcdfc0ffbe0 2023-07-14T11:59:48.009 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.026+0000 7fcdfaffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcdfc100dd0 0x7fcdfc1980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:48.009 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.026+0000 7fcdfaffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcdfc100dd0 0x7fcdfc1980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.17:53640/0 (socket says 172.21.15.17:53640) 2023-07-14T11:59:48.009 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.026+0000 7fcdfbfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdfc101760 0x7fcdfc19f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:48.010 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.026+0000 7fcdfaffd700 1 -- 172.21.15.17:0/3958577332 learned_addr learned my addr 172.21.15.17:0/3958577332 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:59:48.010 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.026+0000 7fcdfb7fe700 1 --2- 172.21.15.17:0/3958577332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdfc0ffbe0 0x7fcdfc197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:48.010 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.026+0000 7fcdfaffd700 1 -- 172.21.15.17:0/3958577332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdfc0ffbe0 msgr2=0x7fcdfc197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:48.010 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.026+0000 7fcdfaffd700 1 --2- 172.21.15.17:0/3958577332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdfc0ffbe0 0x7fcdfc197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:48.010 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.026+0000 7fcdfaffd700 1 -- 172.21.15.17:0/3958577332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdfc101760 msgr2=0x7fcdfc19f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:48.010 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.026+0000 7fcdfaffd700 1 --2- 172.21.15.17:0/3958577332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdfc101760 0x7fcdfc19f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:48.011 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.026+0000 7fcdfaffd700 1 -- 172.21.15.17:0/3958577332 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdec0095f0 con 0x7fcdfc100dd0 2023-07-14T11:59:48.011 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.026+0000 7fcdfbfff700 1 --2- 172.21.15.17:0/3958577332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdfc101760 0x7fcdfc19f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:59:48.011 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.026+0000 7fcdfaffd700 1 --2- 172.21.15.17:0/3958577332 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcdfc100dd0 0x7fcdfc1980f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcdf000e9a0 tx=0x7fcdf000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:48.011 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.027+0000 7fcdf8ff9700 1 -- 172.21.15.17:0/3958577332 <== mon.2 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0009790 con 0x7fcdfc100dd0 2023-07-14T11:59:48.011 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.027+0000 7fcdf8ff9700 1 -- 172.21.15.17:0/3958577332 <== mon.2 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcdf0010430 con 0x7fcdfc100dd0 2023-07-14T11:59:48.011 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.027+0000 7fce01dff700 1 -- 172.21.15.17:0/3958577332 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdfc19fce0 con 0x7fcdfc100dd0 2023-07-14T11:59:48.011 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.027+0000 7fcdf8ff9700 1 -- 172.21.15.17:0/3958577332 <== mon.2 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0018680 con 0x7fcdfc100dd0 2023-07-14T11:59:48.012 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.027+0000 7fce01dff700 1 -- 172.21.15.17:0/3958577332 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdfc1a01a0 con 0x7fcdfc100dd0 2023-07-14T11:59:48.013 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.028+0000 7fce01dff700 1 -- 172.21.15.17:0/3958577332 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdfc1922c0 con 0x7fcdfc100dd0 2023-07-14T11:59:48.013 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.030+0000 7fcdf8ff9700 1 -- 172.21.15.17:0/3958577332 <== mon.2 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fcdf00105a0 con 0x7fcdfc100dd0 2023-07-14T11:59:48.013 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.031+0000 7fcdf8ff9700 1 --2- 172.21.15.17:0/3958577332 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fcde40a1150 0x7fcde40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:48.013 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.032+0000 7fcdf8ff9700 1 -- 172.21.15.17:0/3958577332 <== mon.2 v2:172.21.15.93:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8698+0+0 (secure 0 0 0) 0x7fcdf0014070 con 0x7fcdfc100dd0 2023-07-14T11:59:48.013 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.032+0000 7fcdfb7fe700 1 --2- 172.21.15.17:0/3958577332 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fcde40a1150 0x7fcde40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:48.014 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.032+0000 7fcdfb7fe700 1 --2- 172.21.15.17:0/3958577332 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fcde40a1150 0x7fcde40a3600 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcdec00b2e0 tx=0x7fcdec005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:48.014 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.033+0000 7fcdf8ff9700 1 -- 172.21.15.17:0/3958577332 <== mon.2 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fcdf0096400 con 0x7fcdfc100dd0 2023-07-14T11:59:48.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:47 smithi093 ceph-mon[128922]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-07-14T11:59:48.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:47 smithi093 ceph-mon[128922]: osdmap e86: 12 total, 12 up, 12 in 2023-07-14T11:59:48.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:47 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/1593466105' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-07-14T11:59:48.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:47 smithi093 ceph-mon[128922]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-07-14T11:59:48.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:47 smithi093 ceph-mon[128922]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-07-14T11:59:48.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:47 smithi093 ceph-mon[128922]: osdmap e87: 12 total, 12 up, 12 in 2023-07-14T11:59:48.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:47 smithi097 ceph-mon[129313]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-07-14T11:59:48.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:47 smithi097 ceph-mon[129313]: osdmap e86: 12 total, 12 up, 12 in 2023-07-14T11:59:48.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:47 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/1593466105' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-07-14T11:59:48.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:47 smithi097 ceph-mon[129313]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-07-14T11:59:48.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:47 smithi097 ceph-mon[129313]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-07-14T11:59:48.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:47 smithi097 ceph-mon[129313]: osdmap e87: 12 total, 12 up, 12 in 2023-07-14T11:59:48.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:47 smithi017 ceph-mon[126339]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-07-14T11:59:48.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:47 smithi017 ceph-mon[126339]: osdmap e86: 12 total, 12 up, 12 in 2023-07-14T11:59:48.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:47 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/1593466105' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-07-14T11:59:48.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:47 smithi017 ceph-mon[126339]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-07-14T11:59:48.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:47 smithi017 ceph-mon[126339]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-07-14T11:59:48.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:47 smithi017 ceph-mon[126339]: osdmap e87: 12 total, 12 up, 12 in 2023-07-14T11:59:48.273 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.293+0000 7fce01dff700 1 -- 172.21.15.17:0/3958577332 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7fcdfc02c850 con 0x7fcdfc100dd0 2023-07-14T11:59:48.783 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.802+0000 7fcdf8ff9700 1 -- 172.21.15.17:0/3958577332 <== mon.2 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7fcdf00260b0 con 0x7fcdfc100dd0 2023-07-14T11:59:48.787 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.806+0000 7fce01dff700 1 -- 172.21.15.17:0/3958577332 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fcde40a1150 msgr2=0x7fcde40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:48.787 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.806+0000 7fce01dff700 1 --2- 172.21.15.17:0/3958577332 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fcde40a1150 0x7fcde40a3600 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcdec00b2e0 tx=0x7fcdec005fb0 comp rx=0 tx=0).stop 2023-07-14T11:59:48.787 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.806+0000 7fce01dff700 1 -- 172.21.15.17:0/3958577332 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcdfc100dd0 msgr2=0x7fcdfc1980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:48.787 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.806+0000 7fce01dff700 1 --2- 172.21.15.17:0/3958577332 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcdfc100dd0 0x7fcdfc1980f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcdf000e9a0 tx=0x7fcdf000ecb0 comp rx=0 tx=0).stop 2023-07-14T11:59:48.787 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.806+0000 7fce01dff700 1 -- 172.21.15.17:0/3958577332 shutdown_connections 2023-07-14T11:59:48.787 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.806+0000 7fce01dff700 1 --2- 172.21.15.17:0/3958577332 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fcde40a1150 0x7fcde40a3600 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:48.788 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.806+0000 7fce01dff700 1 --2- 172.21.15.17:0/3958577332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdfc101760 0x7fcdfc19f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:48.788 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.806+0000 7fce01dff700 1 --2- 172.21.15.17:0/3958577332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdfc0ffbe0 0x7fcdfc197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:48.788 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.806+0000 7fce01dff700 1 --2- 172.21.15.17:0/3958577332 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcdfc100dd0 0x7fcdfc1980f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:48.788 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.806+0000 7fce01dff700 1 -- 172.21.15.17:0/3958577332 >> 172.21.15.17:0/3958577332 conn(0x7fcdfc0fb200 msgr2=0x7fcdfc1066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:48.788 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.806+0000 7fce01dff700 1 -- 172.21.15.17:0/3958577332 shutdown_connections 2023-07-14T11:59:48.788 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:48.806+0000 7fce01dff700 1 -- 172.21.15.17:0/3958577332 wait complete. 2023-07-14T11:59:48.789 INFO:teuthology.orchestra.run.smithi017.stderr:added data pool 4 to fsmap 2023-07-14T11:59:48.809 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:59:49.020 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.039+0000 7fd29afff700 1 -- 172.21.15.17:0/3191181757 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd294072ee0 msgr2=0x7fd294073350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:49.020 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.039+0000 7fd29afff700 1 --2- 172.21.15.17:0/3191181757 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd294072ee0 0x7fd294073350 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd280009990 tx=0x7fd280009ca0 comp rx=0 tx=0).stop 2023-07-14T11:59:49.020 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.039+0000 7fd29afff700 1 -- 172.21.15.17:0/3191181757 shutdown_connections 2023-07-14T11:59:49.020 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.039+0000 7fd29afff700 1 --2- 172.21.15.17:0/3191181757 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd2940744b0 0x7fd2940729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:49.020 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.039+0000 7fd29afff700 1 --2- 172.21.15.17:0/3191181757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd294107a30 0x7fd294109e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:49.021 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.039+0000 7fd29afff700 1 --2- 172.21.15.17:0/3191181757 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd294072ee0 0x7fd294073350 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:49.021 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.039+0000 7fd29afff700 1 -- 172.21.15.17:0/3191181757 >> 172.21.15.17:0/3191181757 conn(0x7fd2940fb1d0 msgr2=0x7fd2940fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:49.021 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.039+0000 7fd29afff700 1 -- 172.21.15.17:0/3191181757 shutdown_connections 2023-07-14T11:59:49.021 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.039+0000 7fd29afff700 1 -- 172.21.15.17:0/3191181757 wait complete. 2023-07-14T11:59:49.023 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.040+0000 7fd29afff700 1 Processor -- start 2023-07-14T11:59:49.023 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.040+0000 7fd29afff700 1 -- start start 2023-07-14T11:59:49.023 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.040+0000 7fd29afff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd294072ee0 0x7fd294197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:49.023 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.040+0000 7fd29afff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd2940744b0 0x7fd2941981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:49.023 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.041+0000 7fd29afff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd294107a30 0x7fd29419f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:49.023 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.041+0000 7fd29afff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29419f810 con 0x7fd294072ee0 2023-07-14T11:59:49.024 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.041+0000 7fd29afff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29419f980 con 0x7fd2940744b0 2023-07-14T11:59:49.024 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.041+0000 7fd29afff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29419fb40 con 0x7fd294107a30 2023-07-14T11:59:49.024 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.041+0000 7fd298d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd294072ee0 0x7fd294197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:49.024 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.041+0000 7fd298d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd294072ee0 0x7fd294197ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51894/0 (socket says 172.21.15.17:51894) 2023-07-14T11:59:49.024 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.041+0000 7fd298d9b700 1 -- 172.21.15.17:0/59338510 learned_addr learned my addr 172.21.15.17:0/59338510 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:59:49.024 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.041+0000 7fd29959c700 1 --2- 172.21.15.17:0/59338510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd294107a30 0x7fd29419f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:49.024 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.041+0000 7fd293fff700 1 --2- 172.21.15.17:0/59338510 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd2940744b0 0x7fd2941981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:49.025 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.041+0000 7fd298d9b700 1 -- 172.21.15.17:0/59338510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd294107a30 msgr2=0x7fd29419f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:49.025 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.041+0000 7fd298d9b700 1 --2- 172.21.15.17:0/59338510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd294107a30 0x7fd29419f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:49.025 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.041+0000 7fd298d9b700 1 -- 172.21.15.17:0/59338510 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd2940744b0 msgr2=0x7fd2941981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:49.025 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.041+0000 7fd298d9b700 1 --2- 172.21.15.17:0/59338510 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd2940744b0 0x7fd2941981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:49.025 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.041+0000 7fd298d9b700 1 -- 172.21.15.17:0/59338510 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2800095f0 con 0x7fd294072ee0 2023-07-14T11:59:49.025 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.041+0000 7fd293fff700 1 --2- 172.21.15.17:0/59338510 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd2940744b0 0x7fd2941981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:59:49.025 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.041+0000 7fd298d9b700 1 --2- 172.21.15.17:0/59338510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd294072ee0 0x7fd294197ca0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd28800e970 tx=0x7fd28800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:49.026 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.042+0000 7fd291ffb700 1 -- 172.21.15.17:0/59338510 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd288009790 con 0x7fd294072ee0 2023-07-14T11:59:49.026 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.042+0000 7fd291ffb700 1 -- 172.21.15.17:0/59338510 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd28800c7e0 con 0x7fd294072ee0 2023-07-14T11:59:49.026 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.042+0000 7fd29afff700 1 -- 172.21.15.17:0/59338510 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd29419fe20 con 0x7fd294072ee0 2023-07-14T11:59:49.026 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.042+0000 7fd291ffb700 1 -- 172.21.15.17:0/59338510 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd288003c70 con 0x7fd294072ee0 2023-07-14T11:59:49.026 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.042+0000 7fd29afff700 1 -- 172.21.15.17:0/59338510 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2941a0240 con 0x7fd294072ee0 2023-07-14T11:59:49.027 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.043+0000 7fd29afff700 1 -- 172.21.15.17:0/59338510 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd29410d600 con 0x7fd294072ee0 2023-07-14T11:59:49.027 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.045+0000 7fd291ffb700 1 -- 172.21.15.17:0/59338510 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd28800cc50 con 0x7fd294072ee0 2023-07-14T11:59:49.027 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.046+0000 7fd291ffb700 1 --2- 172.21.15.17:0/59338510 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fd2840a1150 0x7fd2840a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:49.027 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.047+0000 7fd291ffb700 1 -- 172.21.15.17:0/59338510 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd288014070 con 0x7fd294072ee0 2023-07-14T11:59:49.028 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.047+0000 7fd293fff700 1 --2- 172.21.15.17:0/59338510 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fd2840a1150 0x7fd2840a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:49.028 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.047+0000 7fd293fff700 1 --2- 172.21.15.17:0/59338510 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fd2840a1150 0x7fd2840a3600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd2941991a0 tx=0x7fd280005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:49.031 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.050+0000 7fd291ffb700 1 -- 172.21.15.17:0/59338510 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd2880961f0 con 0x7fd294072ee0 2023-07-14T11:59:49.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:48 smithi093 ceph-mon[128922]: pgmap v277: 129 pgs: 27 creating+peering, 81 active+clean, 21 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 9 op/s 2023-07-14T11:59:49.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:48 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/3958577332' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-07-14T11:59:49.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:48 smithi093 ceph-mon[128922]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-07-14T11:59:49.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:48 smithi093 ceph-mon[128922]: osdmap e88: 12 total, 12 up, 12 in 2023-07-14T11:59:49.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:48 smithi097 ceph-mon[129313]: pgmap v277: 129 pgs: 27 creating+peering, 81 active+clean, 21 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 9 op/s 2023-07-14T11:59:49.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:48 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/3958577332' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-07-14T11:59:49.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:48 smithi097 ceph-mon[129313]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-07-14T11:59:49.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:48 smithi097 ceph-mon[129313]: osdmap e88: 12 total, 12 up, 12 in 2023-07-14T11:59:49.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:48 smithi017 ceph-mon[126339]: pgmap v277: 129 pgs: 27 creating+peering, 81 active+clean, 21 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 9 op/s 2023-07-14T11:59:49.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:48 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/3958577332' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-07-14T11:59:49.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:48 smithi017 ceph-mon[126339]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-07-14T11:59:49.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:48 smithi017 ceph-mon[126339]: osdmap e88: 12 total, 12 up, 12 in 2023-07-14T11:59:49.296 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.315+0000 7fd29afff700 1 -- 172.21.15.17:0/59338510 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2941061e0 con 0x7fd294072ee0 2023-07-14T11:59:49.296 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.316+0000 7fd291ffb700 1 -- 172.21.15.17:0/59338510 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+10758 (secure 0 0 0) 0x7fd288099810 con 0x7fd294072ee0 2023-07-14T11:59:49.297 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:59:49.297 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/55384193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":55384193},{"type":"v1","addr":"172.21.15.17:6835","nonce":55384193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/424956554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":424956554},{"type":"v1","addr":"172.21.15.17:6837","nonce":424956554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/326319712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":326319712},{"type":"v1","addr":"172.21.15.93:6833","nonce":326319712}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/878913679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":878913679},{"type":"v1","addr":"172.21.15.97:6833","nonce":878913679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/2779420503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":2779420503},{"type":"v1","addr":"172.21.15.93:6835","nonce":2779420503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2740799393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2740799393},{"type":"v1","addr":"172.21.15.97:6835","nonce":2740799393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24509,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/1880673911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":1880673911},{"type":"v1","addr":"172.21.15.93:6837","nonce":1880673911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3684187131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3684187131},{"type":"v1","addr":"172.21.15.97:6837","nonce":3684187131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24533,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6839/2976493015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":2976493015},{"type":"v1","addr":"172.21.15.17:6839","nonce":2976493015}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24541,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/3267815331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3267815331},{"type":"v1","addr":"172.21.15.97:6839","nonce":3267815331}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6839/1281342428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":1281342428},{"type":"v1","addr":"172.21.15.93:6839","nonce":1281342428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-14T11:59:43.292866+0000","modified":"2023-07-14T11:59:48.798071+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24565},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24565":{"gid":24565,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6841/3905201143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6840","nonce":3905201143},{"type":"v1","addr":"172.21.15.17:6841","nonce":3905201143}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-07-14T11:59:49.299 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.318+0000 7fd29afff700 1 -- 172.21.15.17:0/59338510 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fd2840a1150 msgr2=0x7fd2840a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:49.299 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.318+0000 7fd29afff700 1 --2- 172.21.15.17:0/59338510 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fd2840a1150 0x7fd2840a3600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd2941991a0 tx=0x7fd280005fb0 comp rx=0 tx=0).stop 2023-07-14T11:59:49.299 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.318+0000 7fd29afff700 1 -- 172.21.15.17:0/59338510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd294072ee0 msgr2=0x7fd294197ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:49.299 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.318+0000 7fd29afff700 1 --2- 172.21.15.17:0/59338510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd294072ee0 0x7fd294197ca0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd28800e970 tx=0x7fd28800ec80 comp rx=0 tx=0).stop 2023-07-14T11:59:49.300 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.318+0000 7fd29afff700 1 -- 172.21.15.17:0/59338510 shutdown_connections 2023-07-14T11:59:49.300 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.318+0000 7fd29afff700 1 --2- 172.21.15.17:0/59338510 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fd2840a1150 0x7fd2840a3600 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:49.300 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.318+0000 7fd29afff700 1 --2- 172.21.15.17:0/59338510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd294107a30 0x7fd29419f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:49.300 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.318+0000 7fd29afff700 1 --2- 172.21.15.17:0/59338510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd294072ee0 0x7fd294197ca0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:49.300 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.318+0000 7fd29afff700 1 --2- 172.21.15.17:0/59338510 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd2940744b0 0x7fd2941981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:49.300 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.318+0000 7fd29afff700 1 -- 172.21.15.17:0/59338510 >> 172.21.15.17:0/59338510 conn(0x7fd2940fb1d0 msgr2=0x7fd2940fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:49.300 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.319+0000 7fd29afff700 1 -- 172.21.15.17:0/59338510 shutdown_connections 2023-07-14T11:59:49.301 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.319+0000 7fd29afff700 1 -- 172.21.15.17:0/59338510 wait complete. 2023-07-14T11:59:49.301 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 17 2023-07-14T11:59:49.318 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:59:49.522 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.541+0000 7fabac7ff700 1 -- 172.21.15.17:0/1756002075 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faba40ffba0 msgr2=0x7faba40fffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:49.522 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.541+0000 7fabac7ff700 1 --2- 172.21.15.17:0/1756002075 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faba40ffba0 0x7faba40fffb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7faba0009990 tx=0x7faba0009ca0 comp rx=0 tx=0).stop 2023-07-14T11:59:49.523 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.541+0000 7fabac7ff700 1 -- 172.21.15.17:0/1756002075 shutdown_connections 2023-07-14T11:59:49.523 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.541+0000 7fabac7ff700 1 --2- 172.21.15.17:0/1756002075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faba41017e0 0x7faba4108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:49.523 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.541+0000 7fabac7ff700 1 --2- 172.21.15.17:0/1756002075 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faba40ffba0 0x7faba40fffb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:49.523 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.541+0000 7fabac7ff700 1 --2- 172.21.15.17:0/1756002075 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faba4100e30 0x7faba41012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:49.523 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.541+0000 7fabac7ff700 1 -- 172.21.15.17:0/1756002075 >> 172.21.15.17:0/1756002075 conn(0x7faba40fb1c0 msgr2=0x7faba40fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:49.523 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.541+0000 7fabac7ff700 1 -- 172.21.15.17:0/1756002075 shutdown_connections 2023-07-14T11:59:49.523 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.541+0000 7fabac7ff700 1 -- 172.21.15.17:0/1756002075 wait complete. 2023-07-14T11:59:49.525 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.543+0000 7fabac7ff700 1 Processor -- start 2023-07-14T11:59:49.525 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.543+0000 7fabac7ff700 1 -- start start 2023-07-14T11:59:49.525 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.544+0000 7fabac7ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faba40ffba0 0x7faba40742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:49.525 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.544+0000 7fabac7ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faba4100e30 0x7faba4072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:49.526 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.544+0000 7fabac7ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faba41017e0 0x7faba4072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:49.526 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.544+0000 7fabac7ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba40735e0 con 0x7faba4100e30 2023-07-14T11:59:49.526 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.544+0000 7fabac7ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba4073750 con 0x7faba41017e0 2023-07-14T11:59:49.526 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.544+0000 7fabac7ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba419f990 con 0x7faba40ffba0 2023-07-14T11:59:49.526 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.544+0000 7fabaa59b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faba40ffba0 0x7faba40742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:49.526 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.544+0000 7faba9d9a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faba4100e30 0x7faba4072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:49.526 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.544+0000 7fabaa59b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faba40ffba0 0x7faba40742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.17:58598/0 (socket says 172.21.15.17:58598) 2023-07-14T11:59:49.527 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.544+0000 7fabaa59b700 1 -- 172.21.15.17:0/410563135 learned_addr learned my addr 172.21.15.17:0/410563135 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:59:49.527 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.544+0000 7fabaad9c700 1 --2- 172.21.15.17:0/410563135 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faba41017e0 0x7faba4072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:49.527 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.544+0000 7fabaa59b700 1 -- 172.21.15.17:0/410563135 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faba41017e0 msgr2=0x7faba4072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:49.527 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.544+0000 7fabaa59b700 1 --2- 172.21.15.17:0/410563135 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faba41017e0 0x7faba4072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:49.527 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.544+0000 7fabaa59b700 1 -- 172.21.15.17:0/410563135 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faba4100e30 msgr2=0x7faba4072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:49.527 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.544+0000 7fabaa59b700 1 --2- 172.21.15.17:0/410563135 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faba4100e30 0x7faba4072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:49.527 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.544+0000 7fabaa59b700 1 -- 172.21.15.17:0/410563135 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab94009520 con 0x7faba40ffba0 2023-07-14T11:59:49.527 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.544+0000 7faba9d9a700 1 --2- 172.21.15.17:0/410563135 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faba4100e30 0x7faba4072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:59:49.528 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.545+0000 7fabaa59b700 1 --2- 172.21.15.17:0/410563135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faba40ffba0 0x7faba40742c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7faba000b680 tx=0x7faba0005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:49.528 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.545+0000 7fab9b7fe700 1 -- 172.21.15.17:0/410563135 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba001d070 con 0x7faba40ffba0 2023-07-14T11:59:49.528 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.545+0000 7fabac7ff700 1 -- 172.21.15.17:0/410563135 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faba00095f0 con 0x7faba40ffba0 2023-07-14T11:59:49.529 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.545+0000 7fab9b7fe700 1 -- 172.21.15.17:0/410563135 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faba0022470 con 0x7faba40ffba0 2023-07-14T11:59:49.529 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.545+0000 7fab9b7fe700 1 -- 172.21.15.17:0/410563135 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba0017650 con 0x7faba40ffba0 2023-07-14T11:59:49.529 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.545+0000 7fabac7ff700 1 -- 172.21.15.17:0/410563135 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faba419ff00 con 0x7faba40ffba0 2023-07-14T11:59:49.529 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.546+0000 7fabac7ff700 1 -- 172.21.15.17:0/410563135 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faba4106bc0 con 0x7faba40ffba0 2023-07-14T11:59:49.529 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.548+0000 7fab9b7fe700 1 -- 172.21.15.17:0/410563135 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7faba00225e0 con 0x7faba40ffba0 2023-07-14T11:59:49.530 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.550+0000 7fab9b7fe700 1 --2- 172.21.15.17:0/410563135 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fab900a1150 0x7fab900a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:49.530 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.550+0000 7fab9b7fe700 1 -- 172.21.15.17:0/410563135 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7faba00ce6b0 con 0x7faba40ffba0 2023-07-14T11:59:49.531 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.550+0000 7faba9d9a700 1 --2- 172.21.15.17:0/410563135 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fab900a1150 0x7fab900a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:49.531 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.550+0000 7faba9d9a700 1 --2- 172.21.15.17:0/410563135 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fab900a1150 0x7fab900a3600 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fab94009fd0 tx=0x7fab94009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:49.532 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.551+0000 7fab9b7fe700 1 -- 172.21.15.17:0/410563135 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7faba0097c40 con 0x7faba40ffba0 2023-07-14T11:59:49.742 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.761+0000 7fabac7ff700 1 -- 172.21.15.17:0/410563135 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faba402c850 con 0x7faba40ffba0 2023-07-14T11:59:49.744 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.763+0000 7fab9b7fe700 1 -- 172.21.15.17:0/410563135 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23866 (secure 0 0 0) 0x7faba009b260 con 0x7faba40ffba0 2023-07-14T11:59:49.744 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:59:49.745 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":88,"fsid":"b3c386cc-223c-11ee-9b33-001a4aab830c","created":"2023-07-14T11:51:59.992960+0000","modified":"2023-07-14T11:59:48.295704+0000","last_up_change":"2023-07-14T11:58:55.037769+0000","last_in_change":"2023-07-14T11:58:37.114870+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:55:28.237002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:59:41.560502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T11:59:42.725461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:59:46.267029+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"57003438-a83c-4c8e-a029-56414740895c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6803","nonce":3588366384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6805","nonce":3588366384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6809","nonce":3588366384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6807","nonce":3588366384}]},"public_addr":"172.21.15.17:6803/3588366384","cluster_addr":"172.21.15.17:6805/3588366384","heartbeat_back_addr":"172.21.15.17:6809/3588366384","heartbeat_front_addr":"172.21.15.17:6807/3588366384","state":["exists","up"]},{"osd":1,"uuid":"f56b6337-d8a9-4411-b9d8-b6863dc98556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6801","nonce":3147203064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6803","nonce":3147203064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6807","nonce":3147203064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6805","nonce":3147203064}]},"public_addr":"172.21.15.93:6801/3147203064","cluster_addr":"172.21.15.93:6803/3147203064","heartbeat_back_addr":"172.21.15.93:6807/3147203064","heartbeat_front_addr":"172.21.15.93:6805/3147203064","state":["exists","up"]},{"osd":2,"uuid":"fc34bb7f-551f-46e1-99c0-59924d6c1346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6801","nonce":1877269815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6803","nonce":1877269815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6807","nonce":1877269815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6805","nonce":1877269815}]},"public_addr":"172.21.15.97:6801/1877269815","cluster_addr":"172.21.15.97:6803/1877269815","heartbeat_back_addr":"172.21.15.97:6807/1877269815","heartbeat_front_addr":"172.21.15.97:6805/1877269815","state":["exists","up"]},{"osd":3,"uuid":"37a1583c-4fb8-412a-a0c8-f83b1ff26a91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6811","nonce":2784619821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6813","nonce":2784619821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6817","nonce":2784619821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6815","nonce":2784619821}]},"public_addr":"172.21.15.17:6811/2784619821","cluster_addr":"172.21.15.17:6813/2784619821","heartbeat_back_addr":"172.21.15.17:6817/2784619821","heartbeat_front_addr":"172.21.15.17:6815/2784619821","state":["exists","up"]},{"osd":4,"uuid":"f4b9f377-d519-4b14-ac4a-f1debaeed9f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6809","nonce":3151801173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6811","nonce":3151801173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6815","nonce":3151801173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6813","nonce":3151801173}]},"public_addr":"172.21.15.93:6809/3151801173","cluster_addr":"172.21.15.93:6811/3151801173","heartbeat_back_addr":"172.21.15.93:6815/3151801173","heartbeat_front_addr":"172.21.15.93:6813/3151801173","state":["exists","up"]},{"osd":5,"uuid":"2d2af354-0535-4ef9-b166-3ef219498db8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6809","nonce":2184413924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6811","nonce":2184413924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6815","nonce":2184413924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6813","nonce":2184413924}]},"public_addr":"172.21.15.97:6809/2184413924","cluster_addr":"172.21.15.97:6811/2184413924","heartbeat_back_addr":"172.21.15.97:6815/2184413924","heartbeat_front_addr":"172.21.15.97:6813/2184413924","state":["exists","up"]},{"osd":6,"uuid":"9cdf0c4b-62dc-4168-864e-a9d1650a9ea8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6819","nonce":50486581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6821","nonce":50486581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6825","nonce":50486581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6823","nonce":50486581}]},"public_addr":"172.21.15.17:6819/50486581","cluster_addr":"172.21.15.17:6821/50486581","heartbeat_back_addr":"172.21.15.17:6825/50486581","heartbeat_front_addr":"172.21.15.17:6823/50486581","state":["exists","up"]},{"osd":7,"uuid":"09f60823-5491-45cd-9324-6ae0d56ce262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6817","nonce":2596912392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6819","nonce":2596912392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6823","nonce":2596912392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6821","nonce":2596912392}]},"public_addr":"172.21.15.93:6817/2596912392","cluster_addr":"172.21.15.93:6819/2596912392","heartbeat_back_addr":"172.21.15.93:6823/2596912392","heartbeat_front_addr":"172.21.15.93:6821/2596912392","state":["exists","up"]},{"osd":8,"uuid":"8c5771c9-718e-4313-a2b6-5e1c37c53f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6817","nonce":1965492859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6819","nonce":1965492859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6823","nonce":1965492859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6821","nonce":1965492859}]},"public_addr":"172.21.15.97:6817/1965492859","cluster_addr":"172.21.15.97:6819/1965492859","heartbeat_back_addr":"172.21.15.97:6823/1965492859","heartbeat_front_addr":"172.21.15.97:6821/1965492859","state":["exists","up"]},{"osd":9,"uuid":"1197b8a3-c09f-4c57-aba1-9e988a1620b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6827","nonce":2512645700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6829","nonce":2512645700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6833","nonce":2512645700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6831","nonce":2512645700}]},"public_addr":"172.21.15.17:6827/2512645700","cluster_addr":"172.21.15.17:6829/2512645700","heartbeat_back_addr":"172.21.15.17:6833/2512645700","heartbeat_front_addr":"172.21.15.17:6831/2512645700","state":["exists","up"]},{"osd":10,"uuid":"f99918c2-cdd8-4a7a-be2e-d1d6e0f828a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6825","nonce":264429558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6827","nonce":264429558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6831","nonce":264429558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6829","nonce":264429558}]},"public_addr":"172.21.15.93:6825/264429558","cluster_addr":"172.21.15.93:6827/264429558","heartbeat_back_addr":"172.21.15.93:6831/264429558","heartbeat_front_addr":"172.21.15.93:6829/264429558","state":["exists","up"]},{"osd":11,"uuid":"6bbb5b79-8d4d-441f-9583-203df6c78cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6825","nonce":3389913754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6827","nonce":3389913754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6831","nonce":3389913754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6829","nonce":3389913754}]},"public_addr":"172.21.15.97:6825/3389913754","cluster_addr":"172.21.15.97:6827/3389913754","heartbeat_back_addr":"172.21.15.97:6831/3389913754","heartbeat_front_addr":"172.21.15.97:6829/3389913754","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:54:41.079618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:03.207899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:24.136961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:50.392609+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:56:12.136237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:56:33.212314+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:00.243646+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:22.059234+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:43.232399+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:10.043896+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:31.910780+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:53.158983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/2084033276":"2023-07-15T11:53:05.639597+0000","172.21.15.17:6801/1059284716":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/3651785636":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/1373538024":"2023-07-15T11:52:37.887946+0000","172.21.15.17:0/3560395020":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6800/1059284716":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/4262912861":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/2258375202":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6800/2804227742":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6801/2804227742":"2023-07-15T11:52:37.887946+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:59:49.746 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.766+0000 7fabac7ff700 1 -- 172.21.15.17:0/410563135 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fab900a1150 msgr2=0x7fab900a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:49.746 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.766+0000 7fabac7ff700 1 --2- 172.21.15.17:0/410563135 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fab900a1150 0x7fab900a3600 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fab94009fd0 tx=0x7fab94009260 comp rx=0 tx=0).stop 2023-07-14T11:59:49.747 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.766+0000 7fabac7ff700 1 -- 172.21.15.17:0/410563135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faba40ffba0 msgr2=0x7faba40742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:49.747 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.766+0000 7fabac7ff700 1 --2- 172.21.15.17:0/410563135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faba40ffba0 0x7faba40742c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7faba000b680 tx=0x7faba0005740 comp rx=0 tx=0).stop 2023-07-14T11:59:49.747 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.766+0000 7fabac7ff700 1 -- 172.21.15.17:0/410563135 shutdown_connections 2023-07-14T11:59:49.747 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.766+0000 7fabac7ff700 1 --2- 172.21.15.17:0/410563135 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fab900a1150 0x7fab900a3600 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:49.747 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.766+0000 7fabac7ff700 1 --2- 172.21.15.17:0/410563135 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faba41017e0 0x7faba4072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:49.747 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.766+0000 7fabac7ff700 1 --2- 172.21.15.17:0/410563135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faba40ffba0 0x7faba40742c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:49.747 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.766+0000 7fabac7ff700 1 --2- 172.21.15.17:0/410563135 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faba4100e30 0x7faba4072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:49.748 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.766+0000 7fabac7ff700 1 -- 172.21.15.17:0/410563135 >> 172.21.15.17:0/410563135 conn(0x7faba40fb1c0 msgr2=0x7faba40fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:49.748 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.766+0000 7fabac7ff700 1 -- 172.21.15.17:0/410563135 shutdown_connections 2023-07-14T11:59:49.748 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.766+0000 7fabac7ff700 1 -- 172.21.15.17:0/410563135 wait complete. 2023-07-14T11:59:49.766 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:59:49.972 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.990+0000 7f5df43ff700 1 -- 172.21.15.17:0/3413378273 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5dec100e30 msgr2=0x7f5dec1012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:49.972 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.990+0000 7f5df43ff700 1 --2- 172.21.15.17:0/3413378273 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5dec100e30 0x7f5dec1012a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc009990 tx=0x7f5ddc009ca0 comp rx=0 tx=0).stop 2023-07-14T11:59:49.973 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.990+0000 7f5df43ff700 1 -- 172.21.15.17:0/3413378273 shutdown_connections 2023-07-14T11:59:49.973 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.990+0000 7f5df43ff700 1 --2- 172.21.15.17:0/3413378273 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5dec0ffba0 0x7f5dec0fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:49.973 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.990+0000 7f5df43ff700 1 --2- 172.21.15.17:0/3413378273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dec1017e0 0x7f5dec108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:49.973 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.990+0000 7f5df43ff700 1 --2- 172.21.15.17:0/3413378273 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5dec100e30 0x7f5dec1012a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:49.973 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.990+0000 7f5df43ff700 1 -- 172.21.15.17:0/3413378273 >> 172.21.15.17:0/3413378273 conn(0x7f5dec0fb1c0 msgr2=0x7f5dec0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:49.973 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.990+0000 7f5df43ff700 1 -- 172.21.15.17:0/3413378273 shutdown_connections 2023-07-14T11:59:49.974 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.990+0000 7f5df43ff700 1 -- 172.21.15.17:0/3413378273 wait complete. 2023-07-14T11:59:49.974 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.991+0000 7f5df43ff700 1 Processor -- start 2023-07-14T11:59:49.974 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.991+0000 7f5df43ff700 1 -- start start 2023-07-14T11:59:49.974 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.991+0000 7f5df43ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dec0ffba0 0x7f5dec197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:49.976 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.991+0000 7f5df43ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5dec100e30 0x7f5dec198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:49.976 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.991+0000 7f5df43ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5dec1017e0 0x7f5dec19f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:49.976 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.991+0000 7f5df43ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dec19f6a0 con 0x7f5dec1017e0 2023-07-14T11:59:49.976 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.991+0000 7f5df43ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dec19f7e0 con 0x7f5dec100e30 2023-07-14T11:59:49.976 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.991+0000 7f5df43ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dec19f970 con 0x7f5dec0ffba0 2023-07-14T11:59:49.976 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.992+0000 7f5df299c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5dec1017e0 0x7f5dec19f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:49.977 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.992+0000 7f5df219b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dec0ffba0 0x7f5dec197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:49.977 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.992+0000 7f5df299c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5dec1017e0 0x7f5dec19f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51928/0 (socket says 172.21.15.17:51928) 2023-07-14T11:59:49.977 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.992+0000 7f5df299c700 1 -- 172.21.15.17:0/3075266480 learned_addr learned my addr 172.21.15.17:0/3075266480 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:59:49.977 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.992+0000 7f5df199a700 1 --2- 172.21.15.17:0/3075266480 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5dec100e30 0x7f5dec198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:49.978 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.992+0000 7f5df299c700 1 -- 172.21.15.17:0/3075266480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dec0ffba0 msgr2=0x7f5dec197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:49.979 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.992+0000 7f5df299c700 1 --2- 172.21.15.17:0/3075266480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dec0ffba0 0x7f5dec197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:49.979 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.992+0000 7f5df299c700 1 -- 172.21.15.17:0/3075266480 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5dec100e30 msgr2=0x7f5dec198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:49.979 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.992+0000 7f5df299c700 1 --2- 172.21.15.17:0/3075266480 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5dec100e30 0x7f5dec198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:49.979 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.992+0000 7f5df299c700 1 -- 172.21.15.17:0/3075266480 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ddc0095f0 con 0x7f5dec1017e0 2023-07-14T11:59:49.979 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.992+0000 7f5df299c700 1 --2- 172.21.15.17:0/3075266480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5dec1017e0 0x7f5dec19f0d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5de400ef30 tx=0x7f5de400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:49.979 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.992+0000 7f5de37fe700 1 -- 172.21.15.17:0/3075266480 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de4018070 con 0x7f5dec1017e0 2023-07-14T11:59:49.980 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.992+0000 7f5de37fe700 1 -- 172.21.15.17:0/3075266480 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5de400f040 con 0x7f5dec1017e0 2023-07-14T11:59:49.980 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.992+0000 7f5df43ff700 1 -- 172.21.15.17:0/3075266480 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dec19fbf0 con 0x7f5dec1017e0 2023-07-14T11:59:49.980 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.992+0000 7f5de37fe700 1 -- 172.21.15.17:0/3075266480 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de400cd00 con 0x7f5dec1017e0 2023-07-14T11:59:49.980 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.993+0000 7f5df43ff700 1 -- 172.21.15.17:0/3075266480 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dec1a0100 con 0x7f5dec1017e0 2023-07-14T11:59:49.980 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.994+0000 7f5df43ff700 1 -- 172.21.15.17:0/3075266480 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dec192280 con 0x7f5dec1017e0 2023-07-14T11:59:49.980 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.996+0000 7f5de37fe700 1 -- 172.21.15.17:0/3075266480 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5de4003a00 con 0x7f5dec1017e0 2023-07-14T11:59:49.980 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.997+0000 7f5de37fe700 1 --2- 172.21.15.17:0/3075266480 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f5dd80a1150 0x7f5dd80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:49.981 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.998+0000 7f5de37fe700 1 -- 172.21.15.17:0/3075266480 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f5de40cc640 con 0x7f5dec1017e0 2023-07-14T11:59:49.981 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.998+0000 7f5df219b700 1 --2- 172.21.15.17:0/3075266480 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f5dd80a1150 0x7f5dd80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:49.981 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:49.998+0000 7f5df219b700 1 --2- 172.21.15.17:0/3075266480 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f5dd80a1150 0x7f5dd80a3600 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5de8009c60 tx=0x7f5de8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:49.982 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.001+0000 7f5de37fe700 1 -- 172.21.15.17:0/3075266480 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5de4095bd0 con 0x7f5dec1017e0 2023-07-14T11:59:50.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:49 smithi093 ceph-mon[128922]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-07-14T11:59:50.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:49 smithi093 ceph-mon[128922]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2023-07-14T11:59:50.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:49 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/59338510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:59:50.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:49 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/410563135' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:59:50.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:49 smithi097 ceph-mon[129313]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-07-14T11:59:50.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:49 smithi097 ceph-mon[129313]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2023-07-14T11:59:50.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:49 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/59338510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:59:50.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:49 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/410563135' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:59:50.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:49 smithi017 ceph-mon[126339]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-07-14T11:59:50.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:49 smithi017 ceph-mon[126339]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2023-07-14T11:59:50.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:49 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/59338510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:59:50.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:49 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/410563135' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:59:50.190 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.210+0000 7f5df43ff700 1 -- 172.21.15.17:0/3075266480 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5dec02c850 con 0x7f5dec1017e0 2023-07-14T11:59:50.192 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.211+0000 7f5de37fe700 1 -- 172.21.15.17:0/3075266480 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23866 (secure 0 0 0) 0x7f5de40991f0 con 0x7f5dec1017e0 2023-07-14T11:59:50.192 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:59:50.193 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":88,"fsid":"b3c386cc-223c-11ee-9b33-001a4aab830c","created":"2023-07-14T11:51:59.992960+0000","modified":"2023-07-14T11:59:48.295704+0000","last_up_change":"2023-07-14T11:58:55.037769+0000","last_in_change":"2023-07-14T11:58:37.114870+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:55:28.237002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:59:41.560502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T11:59:42.725461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:59:46.267029+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"57003438-a83c-4c8e-a029-56414740895c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6803","nonce":3588366384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6805","nonce":3588366384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6809","nonce":3588366384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6807","nonce":3588366384}]},"public_addr":"172.21.15.17:6803/3588366384","cluster_addr":"172.21.15.17:6805/3588366384","heartbeat_back_addr":"172.21.15.17:6809/3588366384","heartbeat_front_addr":"172.21.15.17:6807/3588366384","state":["exists","up"]},{"osd":1,"uuid":"f56b6337-d8a9-4411-b9d8-b6863dc98556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6801","nonce":3147203064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6803","nonce":3147203064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6807","nonce":3147203064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6805","nonce":3147203064}]},"public_addr":"172.21.15.93:6801/3147203064","cluster_addr":"172.21.15.93:6803/3147203064","heartbeat_back_addr":"172.21.15.93:6807/3147203064","heartbeat_front_addr":"172.21.15.93:6805/3147203064","state":["exists","up"]},{"osd":2,"uuid":"fc34bb7f-551f-46e1-99c0-59924d6c1346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6801","nonce":1877269815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6803","nonce":1877269815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6807","nonce":1877269815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6805","nonce":1877269815}]},"public_addr":"172.21.15.97:6801/1877269815","cluster_addr":"172.21.15.97:6803/1877269815","heartbeat_back_addr":"172.21.15.97:6807/1877269815","heartbeat_front_addr":"172.21.15.97:6805/1877269815","state":["exists","up"]},{"osd":3,"uuid":"37a1583c-4fb8-412a-a0c8-f83b1ff26a91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6811","nonce":2784619821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6813","nonce":2784619821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6817","nonce":2784619821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6815","nonce":2784619821}]},"public_addr":"172.21.15.17:6811/2784619821","cluster_addr":"172.21.15.17:6813/2784619821","heartbeat_back_addr":"172.21.15.17:6817/2784619821","heartbeat_front_addr":"172.21.15.17:6815/2784619821","state":["exists","up"]},{"osd":4,"uuid":"f4b9f377-d519-4b14-ac4a-f1debaeed9f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6809","nonce":3151801173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6811","nonce":3151801173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6815","nonce":3151801173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6813","nonce":3151801173}]},"public_addr":"172.21.15.93:6809/3151801173","cluster_addr":"172.21.15.93:6811/3151801173","heartbeat_back_addr":"172.21.15.93:6815/3151801173","heartbeat_front_addr":"172.21.15.93:6813/3151801173","state":["exists","up"]},{"osd":5,"uuid":"2d2af354-0535-4ef9-b166-3ef219498db8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6809","nonce":2184413924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6811","nonce":2184413924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6815","nonce":2184413924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6813","nonce":2184413924}]},"public_addr":"172.21.15.97:6809/2184413924","cluster_addr":"172.21.15.97:6811/2184413924","heartbeat_back_addr":"172.21.15.97:6815/2184413924","heartbeat_front_addr":"172.21.15.97:6813/2184413924","state":["exists","up"]},{"osd":6,"uuid":"9cdf0c4b-62dc-4168-864e-a9d1650a9ea8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6819","nonce":50486581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6821","nonce":50486581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6825","nonce":50486581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6823","nonce":50486581}]},"public_addr":"172.21.15.17:6819/50486581","cluster_addr":"172.21.15.17:6821/50486581","heartbeat_back_addr":"172.21.15.17:6825/50486581","heartbeat_front_addr":"172.21.15.17:6823/50486581","state":["exists","up"]},{"osd":7,"uuid":"09f60823-5491-45cd-9324-6ae0d56ce262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6817","nonce":2596912392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6819","nonce":2596912392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6823","nonce":2596912392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6821","nonce":2596912392}]},"public_addr":"172.21.15.93:6817/2596912392","cluster_addr":"172.21.15.93:6819/2596912392","heartbeat_back_addr":"172.21.15.93:6823/2596912392","heartbeat_front_addr":"172.21.15.93:6821/2596912392","state":["exists","up"]},{"osd":8,"uuid":"8c5771c9-718e-4313-a2b6-5e1c37c53f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6817","nonce":1965492859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6819","nonce":1965492859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6823","nonce":1965492859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6821","nonce":1965492859}]},"public_addr":"172.21.15.97:6817/1965492859","cluster_addr":"172.21.15.97:6819/1965492859","heartbeat_back_addr":"172.21.15.97:6823/1965492859","heartbeat_front_addr":"172.21.15.97:6821/1965492859","state":["exists","up"]},{"osd":9,"uuid":"1197b8a3-c09f-4c57-aba1-9e988a1620b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6827","nonce":2512645700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6829","nonce":2512645700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6833","nonce":2512645700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6831","nonce":2512645700}]},"public_addr":"172.21.15.17:6827/2512645700","cluster_addr":"172.21.15.17:6829/2512645700","heartbeat_back_addr":"172.21.15.17:6833/2512645700","heartbeat_front_addr":"172.21.15.17:6831/2512645700","state":["exists","up"]},{"osd":10,"uuid":"f99918c2-cdd8-4a7a-be2e-d1d6e0f828a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6825","nonce":264429558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6827","nonce":264429558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6831","nonce":264429558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6829","nonce":264429558}]},"public_addr":"172.21.15.93:6825/264429558","cluster_addr":"172.21.15.93:6827/264429558","heartbeat_back_addr":"172.21.15.93:6831/264429558","heartbeat_front_addr":"172.21.15.93:6829/264429558","state":["exists","up"]},{"osd":11,"uuid":"6bbb5b79-8d4d-441f-9583-203df6c78cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6825","nonce":3389913754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6827","nonce":3389913754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6831","nonce":3389913754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6829","nonce":3389913754}]},"public_addr":"172.21.15.97:6825/3389913754","cluster_addr":"172.21.15.97:6827/3389913754","heartbeat_back_addr":"172.21.15.97:6831/3389913754","heartbeat_front_addr":"172.21.15.97:6829/3389913754","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:54:41.079618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:03.207899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:24.136961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:50.392609+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:56:12.136237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:56:33.212314+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:00.243646+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:22.059234+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:43.232399+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:10.043896+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:31.910780+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:53.158983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/2084033276":"2023-07-15T11:53:05.639597+0000","172.21.15.17:6801/1059284716":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/3651785636":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/1373538024":"2023-07-15T11:52:37.887946+0000","172.21.15.17:0/3560395020":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6800/1059284716":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/4262912861":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/2258375202":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6800/2804227742":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6801/2804227742":"2023-07-15T11:52:37.887946+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:59:50.194 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.214+0000 7f5df43ff700 1 -- 172.21.15.17:0/3075266480 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f5dd80a1150 msgr2=0x7f5dd80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:50.194 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.214+0000 7f5df43ff700 1 --2- 172.21.15.17:0/3075266480 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f5dd80a1150 0x7f5dd80a3600 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5de8009c60 tx=0x7f5de8009260 comp rx=0 tx=0).stop 2023-07-14T11:59:50.195 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.214+0000 7f5df43ff700 1 -- 172.21.15.17:0/3075266480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5dec1017e0 msgr2=0x7f5dec19f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:50.195 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.214+0000 7f5df43ff700 1 --2- 172.21.15.17:0/3075266480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5dec1017e0 0x7f5dec19f0d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5de400ef30 tx=0x7f5de400c3a0 comp rx=0 tx=0).stop 2023-07-14T11:59:50.195 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.214+0000 7f5df43ff700 1 -- 172.21.15.17:0/3075266480 shutdown_connections 2023-07-14T11:59:50.195 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.214+0000 7f5df43ff700 1 --2- 172.21.15.17:0/3075266480 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f5dd80a1150 0x7f5dd80a3600 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:50.195 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.214+0000 7f5df43ff700 1 --2- 172.21.15.17:0/3075266480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5dec1017e0 0x7f5dec19f0d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:50.195 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.214+0000 7f5df43ff700 1 --2- 172.21.15.17:0/3075266480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dec0ffba0 0x7f5dec197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:50.195 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.214+0000 7f5df43ff700 1 --2- 172.21.15.17:0/3075266480 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5dec100e30 0x7f5dec198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:50.196 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.214+0000 7f5df43ff700 1 -- 172.21.15.17:0/3075266480 >> 172.21.15.17:0/3075266480 conn(0x7f5dec0fb1c0 msgr2=0x7f5dec106730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:50.196 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.214+0000 7f5df43ff700 1 -- 172.21.15.17:0/3075266480 shutdown_connections 2023-07-14T11:59:50.196 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.214+0000 7f5df43ff700 1 -- 172.21.15.17:0/3075266480 wait complete. 2023-07-14T11:59:50.229 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T11:59:50.433 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.452+0000 7f3ef95ff700 1 -- 172.21.15.17:0/2568265843 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ef4072ee0 msgr2=0x7f3ef4073350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:50.433 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.452+0000 7f3ef95ff700 1 --2- 172.21.15.17:0/2568265843 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ef4072ee0 0x7f3ef4073350 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3edc009990 tx=0x7f3edc009ca0 comp rx=0 tx=0).stop 2023-07-14T11:59:50.433 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.452+0000 7f3ef95ff700 1 -- 172.21.15.17:0/2568265843 shutdown_connections 2023-07-14T11:59:50.434 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.452+0000 7f3ef95ff700 1 --2- 172.21.15.17:0/2568265843 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3ef4107a20 0x7f3ef4109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:50.434 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.452+0000 7f3ef95ff700 1 --2- 172.21.15.17:0/2568265843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ef40744b0 0x7f3ef40729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:50.434 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.452+0000 7f3ef95ff700 1 --2- 172.21.15.17:0/2568265843 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ef4072ee0 0x7f3ef4073350 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:50.434 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.452+0000 7f3ef95ff700 1 -- 172.21.15.17:0/2568265843 >> 172.21.15.17:0/2568265843 conn(0x7f3ef40fb1c0 msgr2=0x7f3ef40fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:50.434 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.452+0000 7f3ef95ff700 1 -- 172.21.15.17:0/2568265843 shutdown_connections 2023-07-14T11:59:50.434 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.452+0000 7f3ef95ff700 1 -- 172.21.15.17:0/2568265843 wait complete. 2023-07-14T11:59:50.436 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.453+0000 7f3ef95ff700 1 Processor -- start 2023-07-14T11:59:50.436 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.453+0000 7f3ef95ff700 1 -- start start 2023-07-14T11:59:50.436 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.454+0000 7f3ef95ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ef4072ee0 0x7f3ef4197cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:50.436 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.454+0000 7f3ef95ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3ef40744b0 0x7f3ef41981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:50.437 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.454+0000 7f3ef95ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ef4107a20 0x7f3ef419f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:50.437 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.454+0000 7f3ef95ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef419f820 con 0x7f3ef4107a20 2023-07-14T11:59:50.437 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.454+0000 7f3ef95ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef419f990 con 0x7f3ef40744b0 2023-07-14T11:59:50.437 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.454+0000 7f3ef95ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef419fb50 con 0x7f3ef4072ee0 2023-07-14T11:59:50.437 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.454+0000 7f3ef37fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ef4107a20 0x7f3ef419f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:50.437 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.454+0000 7f3ef37fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ef4107a20 0x7f3ef419f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51944/0 (socket says 172.21.15.17:51944) 2023-07-14T11:59:50.438 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.454+0000 7f3ef37fe700 1 -- 172.21.15.17:0/3254935930 learned_addr learned my addr 172.21.15.17:0/3254935930 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:59:50.438 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.454+0000 7f3eebfff700 1 --2- 172.21.15.17:0/3254935930 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3ef40744b0 0x7f3ef41981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:50.438 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.454+0000 7f3ef2ffd700 1 --2- 172.21.15.17:0/3254935930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ef4072ee0 0x7f3ef4197cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:50.438 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.454+0000 7f3ef37fe700 1 -- 172.21.15.17:0/3254935930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ef4072ee0 msgr2=0x7f3ef4197cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:50.438 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.454+0000 7f3ef37fe700 1 --2- 172.21.15.17:0/3254935930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ef4072ee0 0x7f3ef4197cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:50.438 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.454+0000 7f3ef37fe700 1 -- 172.21.15.17:0/3254935930 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3ef40744b0 msgr2=0x7f3ef41981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:50.439 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.454+0000 7f3ef37fe700 1 --2- 172.21.15.17:0/3254935930 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3ef40744b0 0x7f3ef41981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:50.439 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.454+0000 7f3ef37fe700 1 -- 172.21.15.17:0/3254935930 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3edc0095f0 con 0x7f3ef4107a20 2023-07-14T11:59:50.439 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.454+0000 7f3eebfff700 1 --2- 172.21.15.17:0/3254935930 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3ef40744b0 0x7f3ef41981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:59:50.439 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.455+0000 7f3ef37fe700 1 --2- 172.21.15.17:0/3254935930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ef4107a20 0x7f3ef419f250 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00ef30 tx=0x7f3eec00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:50.439 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.455+0000 7f3ef0ff9700 1 -- 172.21.15.17:0/3254935930 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eec018070 con 0x7f3ef4107a20 2023-07-14T11:59:50.439 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.455+0000 7f3ef0ff9700 1 -- 172.21.15.17:0/3254935930 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3eec00f040 con 0x7f3ef4107a20 2023-07-14T11:59:50.439 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.455+0000 7f3ef95ff700 1 -- 172.21.15.17:0/3254935930 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ef419fdd0 con 0x7f3ef4107a20 2023-07-14T11:59:50.440 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.455+0000 7f3ef0ff9700 1 -- 172.21.15.17:0/3254935930 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eec00cd00 con 0x7f3ef4107a20 2023-07-14T11:59:50.440 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.455+0000 7f3ef95ff700 1 -- 172.21.15.17:0/3254935930 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ef41a02b0 con 0x7f3ef4107a20 2023-07-14T11:59:50.440 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.456+0000 7f3ef95ff700 1 -- 172.21.15.17:0/3254935930 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ef410d5f0 con 0x7f3ef4107a20 2023-07-14T11:59:50.441 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.458+0000 7f3ef0ff9700 1 -- 172.21.15.17:0/3254935930 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3eec003a00 con 0x7f3ef4107a20 2023-07-14T11:59:50.441 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.460+0000 7f3ef0ff9700 1 --2- 172.21.15.17:0/3254935930 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3ee00a11d0 0x7f3ee00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:50.441 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.460+0000 7f3ef0ff9700 1 -- 172.21.15.17:0/3254935930 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f3eec0cc5c0 con 0x7f3ef4107a20 2023-07-14T11:59:50.441 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.460+0000 7f3ef2ffd700 1 --2- 172.21.15.17:0/3254935930 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3ee00a11d0 0x7f3ee00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:50.441 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.461+0000 7f3ef2ffd700 1 --2- 172.21.15.17:0/3254935930 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3ee00a11d0 0x7f3ee00a3680 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4198b80 tx=0x7f3ee4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:50.444 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.463+0000 7f3ef0ff9700 1 -- 172.21.15.17:0/3254935930 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3eec095b50 con 0x7f3ef4107a20 2023-07-14T11:59:50.702 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.721+0000 7f3ef95ff700 1 -- 172.21.15.17:0/3254935930 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ef410d7e0 con 0x7f3ef4107a20 2023-07-14T11:59:50.702 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.722+0000 7f3ef0ff9700 1 -- 172.21.15.17:0/3254935930 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+10758 (secure 0 0 0) 0x7f3eec099170 con 0x7f3ef4107a20 2023-07-14T11:59:50.703 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:59:50.703 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/55384193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":55384193},{"type":"v1","addr":"172.21.15.17:6835","nonce":55384193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/424956554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":424956554},{"type":"v1","addr":"172.21.15.17:6837","nonce":424956554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/326319712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":326319712},{"type":"v1","addr":"172.21.15.93:6833","nonce":326319712}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/878913679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":878913679},{"type":"v1","addr":"172.21.15.97:6833","nonce":878913679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/2779420503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":2779420503},{"type":"v1","addr":"172.21.15.93:6835","nonce":2779420503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2740799393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2740799393},{"type":"v1","addr":"172.21.15.97:6835","nonce":2740799393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24509,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/1880673911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":1880673911},{"type":"v1","addr":"172.21.15.93:6837","nonce":1880673911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3684187131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3684187131},{"type":"v1","addr":"172.21.15.97:6837","nonce":3684187131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24533,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6839/2976493015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":2976493015},{"type":"v1","addr":"172.21.15.17:6839","nonce":2976493015}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24541,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/3267815331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3267815331},{"type":"v1","addr":"172.21.15.97:6839","nonce":3267815331}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6839/1281342428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":1281342428},{"type":"v1","addr":"172.21.15.93:6839","nonce":1281342428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-14T11:59:43.292866+0000","modified":"2023-07-14T11:59:48.798071+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24565},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24565":{"gid":24565,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6841/3905201143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6840","nonce":3905201143},{"type":"v1","addr":"172.21.15.17:6841","nonce":3905201143}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-07-14T11:59:50.705 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.724+0000 7f3ef95ff700 1 -- 172.21.15.17:0/3254935930 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3ee00a11d0 msgr2=0x7f3ee00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:50.705 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.724+0000 7f3ef95ff700 1 --2- 172.21.15.17:0/3254935930 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3ee00a11d0 0x7f3ee00a3680 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4198b80 tx=0x7f3ee4009260 comp rx=0 tx=0).stop 2023-07-14T11:59:50.705 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.724+0000 7f3ef95ff700 1 -- 172.21.15.17:0/3254935930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ef4107a20 msgr2=0x7f3ef419f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:50.706 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.724+0000 7f3ef95ff700 1 --2- 172.21.15.17:0/3254935930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ef4107a20 0x7f3ef419f250 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00ef30 tx=0x7f3eec00c3a0 comp rx=0 tx=0).stop 2023-07-14T11:59:50.706 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.724+0000 7f3ef95ff700 1 -- 172.21.15.17:0/3254935930 shutdown_connections 2023-07-14T11:59:50.706 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.724+0000 7f3ef95ff700 1 --2- 172.21.15.17:0/3254935930 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3ee00a11d0 0x7f3ee00a3680 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:50.706 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.724+0000 7f3ef95ff700 1 --2- 172.21.15.17:0/3254935930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ef4107a20 0x7f3ef419f250 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:50.706 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.724+0000 7f3ef95ff700 1 --2- 172.21.15.17:0/3254935930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ef4072ee0 0x7f3ef4197cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:50.706 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.724+0000 7f3ef95ff700 1 --2- 172.21.15.17:0/3254935930 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3ef40744b0 0x7f3ef41981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:50.706 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.724+0000 7f3ef95ff700 1 -- 172.21.15.17:0/3254935930 >> 172.21.15.17:0/3254935930 conn(0x7f3ef40fb1c0 msgr2=0x7f3ef40fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:50.706 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.725+0000 7f3ef95ff700 1 -- 172.21.15.17:0/3254935930 shutdown_connections 2023-07-14T11:59:50.707 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.725+0000 7f3ef95ff700 1 -- 172.21.15.17:0/3254935930 wait complete. 2023-07-14T11:59:50.707 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 17 2023-07-14T11:59:50.724 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:59:50.930 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.950+0000 7f5c771ff700 1 -- 172.21.15.17:0/1268663795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c701017f0 msgr2=0x7f5c70108090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:50.931 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.950+0000 7f5c771ff700 1 --2- 172.21.15.17:0/1268663795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c701017f0 0x7f5c70108090 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5c6800b0a0 tx=0x7f5c6800b3b0 comp rx=0 tx=0).stop 2023-07-14T11:59:50.931 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.950+0000 7f5c771ff700 1 -- 172.21.15.17:0/1268663795 shutdown_connections 2023-07-14T11:59:50.931 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.950+0000 7f5c771ff700 1 --2- 172.21.15.17:0/1268663795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c701017f0 0x7f5c70108090 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:50.931 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.950+0000 7f5c771ff700 1 --2- 172.21.15.17:0/1268663795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c700ffbb0 0x7f5c700fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:50.932 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.950+0000 7f5c771ff700 1 --2- 172.21.15.17:0/1268663795 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c70100e40 0x7f5c701012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:50.932 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.950+0000 7f5c771ff700 1 -- 172.21.15.17:0/1268663795 >> 172.21.15.17:0/1268663795 conn(0x7f5c700fb1d0 msgr2=0x7f5c700fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:50.932 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.950+0000 7f5c771ff700 1 -- 172.21.15.17:0/1268663795 shutdown_connections 2023-07-14T11:59:50.932 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.950+0000 7f5c771ff700 1 -- 172.21.15.17:0/1268663795 wait complete. 2023-07-14T11:59:50.934 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.951+0000 7f5c771ff700 1 Processor -- start 2023-07-14T11:59:50.934 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.951+0000 7f5c771ff700 1 -- start start 2023-07-14T11:59:50.934 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.951+0000 7f5c771ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c700ffbb0 0x7f5c700742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:50.934 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.951+0000 7f5c771ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c70100e40 0x7f5c70072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:50.935 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.951+0000 7f5c771ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c701017f0 0x7f5c70072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:50.935 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.951+0000 7f5c771ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c700735e0 con 0x7f5c70100e40 2023-07-14T11:59:50.935 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.951+0000 7f5c771ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c70073750 con 0x7f5c700ffbb0 2023-07-14T11:59:50.935 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.951+0000 7f5c771ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7019fa60 con 0x7f5c701017f0 2023-07-14T11:59:50.935 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.951+0000 7f5c6ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c70100e40 0x7f5c70072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:50.935 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.951+0000 7f5c6ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c70100e40 0x7f5c70072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51952/0 (socket says 172.21.15.17:51952) 2023-07-14T11:59:50.935 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.951+0000 7f5c6ffff700 1 -- 172.21.15.17:0/2243082430 learned_addr learned my addr 172.21.15.17:0/2243082430 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:59:50.935 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.952+0000 7f5c74f9b700 1 --2- 172.21.15.17:0/2243082430 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c700ffbb0 0x7f5c700742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:50.936 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.952+0000 7f5c7579c700 1 --2- 172.21.15.17:0/2243082430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c701017f0 0x7f5c70072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:50.936 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.952+0000 7f5c6ffff700 1 -- 172.21.15.17:0/2243082430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c701017f0 msgr2=0x7f5c70072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:50.936 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.952+0000 7f5c6ffff700 1 --2- 172.21.15.17:0/2243082430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c701017f0 0x7f5c70072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:50.936 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.952+0000 7f5c6ffff700 1 -- 172.21.15.17:0/2243082430 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c700ffbb0 msgr2=0x7f5c700742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:50.936 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.952+0000 7f5c6ffff700 1 --2- 172.21.15.17:0/2243082430 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c700ffbb0 0x7f5c700742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:50.936 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.952+0000 7f5c6ffff700 1 -- 172.21.15.17:0/2243082430 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c68009d00 con 0x7f5c70100e40 2023-07-14T11:59:50.936 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.952+0000 7f5c6ffff700 1 --2- 172.21.15.17:0/2243082430 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c70100e40 0x7f5c70072900 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c00d980 tx=0x7f5c5c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:50.937 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.952+0000 7f5c6dffb700 1 -- 172.21.15.17:0/2243082430 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c004220 con 0x7f5c70100e40 2023-07-14T11:59:50.937 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.952+0000 7f5c6dffb700 1 -- 172.21.15.17:0/2243082430 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c5c007640 con 0x7f5c70100e40 2023-07-14T11:59:50.937 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.952+0000 7f5c771ff700 1 -- 172.21.15.17:0/2243082430 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c7019fd40 con 0x7f5c70100e40 2023-07-14T11:59:50.937 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.952+0000 7f5c6dffb700 1 -- 172.21.15.17:0/2243082430 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c005690 con 0x7f5c70100e40 2023-07-14T11:59:50.937 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.953+0000 7f5c771ff700 1 -- 172.21.15.17:0/2243082430 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c701a01d0 con 0x7f5c70100e40 2023-07-14T11:59:50.938 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.953+0000 7f5c771ff700 1 -- 172.21.15.17:0/2243082430 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c70107190 con 0x7f5c70100e40 2023-07-14T11:59:50.938 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.956+0000 7f5c6dffb700 1 -- 172.21.15.17:0/2243082430 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5c5c004750 con 0x7f5c70100e40 2023-07-14T11:59:50.938 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.958+0000 7f5c6dffb700 1 --2- 172.21.15.17:0/2243082430 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f5c600a1150 0x7f5c600a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:50.939 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.958+0000 7f5c6dffb700 1 -- 172.21.15.17:0/2243082430 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f5c5c0ccde0 con 0x7f5c70100e40 2023-07-14T11:59:50.939 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.958+0000 7f5c74f9b700 1 --2- 172.21.15.17:0/2243082430 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f5c600a1150 0x7f5c600a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:50.940 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.959+0000 7f5c74f9b700 1 --2- 172.21.15.17:0/2243082430 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f5c600a1150 0x7f5c600a3600 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5c64009c60 tx=0x7f5c64009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:50.941 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:50.961+0000 7f5c6dffb700 1 -- 172.21.15.17:0/2243082430 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5c5c096340 con 0x7f5c70100e40 2023-07-14T11:59:51.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:50 smithi093 ceph-mon[128922]: pgmap v280: 129 pgs: 11 creating+peering, 109 active+clean, 9 unknown; 579 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 9 op/s 2023-07-14T11:59:51.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:50 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/3075266480' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:59:51.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:50 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/3254935930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:59:51.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:50 smithi097 ceph-mon[129313]: pgmap v280: 129 pgs: 11 creating+peering, 109 active+clean, 9 unknown; 579 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 9 op/s 2023-07-14T11:59:51.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:50 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/3075266480' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:59:51.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:50 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/3254935930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:59:51.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:50 smithi017 ceph-mon[126339]: pgmap v280: 129 pgs: 11 creating+peering, 109 active+clean, 9 unknown; 579 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 9 op/s 2023-07-14T11:59:51.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:50 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/3075266480' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:59:51.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:50 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/3254935930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T11:59:51.150 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.170+0000 7f5c771ff700 1 -- 172.21.15.17:0/2243082430 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c7002caa0 con 0x7f5c70100e40 2023-07-14T11:59:51.152 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.171+0000 7f5c6dffb700 1 -- 172.21.15.17:0/2243082430 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23866 (secure 0 0 0) 0x7f5c5c016020 con 0x7f5c70100e40 2023-07-14T11:59:51.152 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:59:51.153 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":88,"fsid":"b3c386cc-223c-11ee-9b33-001a4aab830c","created":"2023-07-14T11:51:59.992960+0000","modified":"2023-07-14T11:59:48.295704+0000","last_up_change":"2023-07-14T11:58:55.037769+0000","last_in_change":"2023-07-14T11:58:37.114870+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:55:28.237002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:59:41.560502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T11:59:42.725461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:59:46.267029+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"57003438-a83c-4c8e-a029-56414740895c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6803","nonce":3588366384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6805","nonce":3588366384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6809","nonce":3588366384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6807","nonce":3588366384}]},"public_addr":"172.21.15.17:6803/3588366384","cluster_addr":"172.21.15.17:6805/3588366384","heartbeat_back_addr":"172.21.15.17:6809/3588366384","heartbeat_front_addr":"172.21.15.17:6807/3588366384","state":["exists","up"]},{"osd":1,"uuid":"f56b6337-d8a9-4411-b9d8-b6863dc98556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6801","nonce":3147203064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6803","nonce":3147203064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6807","nonce":3147203064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6805","nonce":3147203064}]},"public_addr":"172.21.15.93:6801/3147203064","cluster_addr":"172.21.15.93:6803/3147203064","heartbeat_back_addr":"172.21.15.93:6807/3147203064","heartbeat_front_addr":"172.21.15.93:6805/3147203064","state":["exists","up"]},{"osd":2,"uuid":"fc34bb7f-551f-46e1-99c0-59924d6c1346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6801","nonce":1877269815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6803","nonce":1877269815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6807","nonce":1877269815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6805","nonce":1877269815}]},"public_addr":"172.21.15.97:6801/1877269815","cluster_addr":"172.21.15.97:6803/1877269815","heartbeat_back_addr":"172.21.15.97:6807/1877269815","heartbeat_front_addr":"172.21.15.97:6805/1877269815","state":["exists","up"]},{"osd":3,"uuid":"37a1583c-4fb8-412a-a0c8-f83b1ff26a91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6811","nonce":2784619821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6813","nonce":2784619821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6817","nonce":2784619821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6815","nonce":2784619821}]},"public_addr":"172.21.15.17:6811/2784619821","cluster_addr":"172.21.15.17:6813/2784619821","heartbeat_back_addr":"172.21.15.17:6817/2784619821","heartbeat_front_addr":"172.21.15.17:6815/2784619821","state":["exists","up"]},{"osd":4,"uuid":"f4b9f377-d519-4b14-ac4a-f1debaeed9f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6809","nonce":3151801173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6811","nonce":3151801173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6815","nonce":3151801173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6813","nonce":3151801173}]},"public_addr":"172.21.15.93:6809/3151801173","cluster_addr":"172.21.15.93:6811/3151801173","heartbeat_back_addr":"172.21.15.93:6815/3151801173","heartbeat_front_addr":"172.21.15.93:6813/3151801173","state":["exists","up"]},{"osd":5,"uuid":"2d2af354-0535-4ef9-b166-3ef219498db8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6809","nonce":2184413924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6811","nonce":2184413924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6815","nonce":2184413924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6813","nonce":2184413924}]},"public_addr":"172.21.15.97:6809/2184413924","cluster_addr":"172.21.15.97:6811/2184413924","heartbeat_back_addr":"172.21.15.97:6815/2184413924","heartbeat_front_addr":"172.21.15.97:6813/2184413924","state":["exists","up"]},{"osd":6,"uuid":"9cdf0c4b-62dc-4168-864e-a9d1650a9ea8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6819","nonce":50486581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6821","nonce":50486581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6825","nonce":50486581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6823","nonce":50486581}]},"public_addr":"172.21.15.17:6819/50486581","cluster_addr":"172.21.15.17:6821/50486581","heartbeat_back_addr":"172.21.15.17:6825/50486581","heartbeat_front_addr":"172.21.15.17:6823/50486581","state":["exists","up"]},{"osd":7,"uuid":"09f60823-5491-45cd-9324-6ae0d56ce262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6817","nonce":2596912392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6819","nonce":2596912392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6823","nonce":2596912392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6821","nonce":2596912392}]},"public_addr":"172.21.15.93:6817/2596912392","cluster_addr":"172.21.15.93:6819/2596912392","heartbeat_back_addr":"172.21.15.93:6823/2596912392","heartbeat_front_addr":"172.21.15.93:6821/2596912392","state":["exists","up"]},{"osd":8,"uuid":"8c5771c9-718e-4313-a2b6-5e1c37c53f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6817","nonce":1965492859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6819","nonce":1965492859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6823","nonce":1965492859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6821","nonce":1965492859}]},"public_addr":"172.21.15.97:6817/1965492859","cluster_addr":"172.21.15.97:6819/1965492859","heartbeat_back_addr":"172.21.15.97:6823/1965492859","heartbeat_front_addr":"172.21.15.97:6821/1965492859","state":["exists","up"]},{"osd":9,"uuid":"1197b8a3-c09f-4c57-aba1-9e988a1620b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6827","nonce":2512645700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6829","nonce":2512645700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6833","nonce":2512645700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6831","nonce":2512645700}]},"public_addr":"172.21.15.17:6827/2512645700","cluster_addr":"172.21.15.17:6829/2512645700","heartbeat_back_addr":"172.21.15.17:6833/2512645700","heartbeat_front_addr":"172.21.15.17:6831/2512645700","state":["exists","up"]},{"osd":10,"uuid":"f99918c2-cdd8-4a7a-be2e-d1d6e0f828a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6825","nonce":264429558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6827","nonce":264429558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6831","nonce":264429558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6829","nonce":264429558}]},"public_addr":"172.21.15.93:6825/264429558","cluster_addr":"172.21.15.93:6827/264429558","heartbeat_back_addr":"172.21.15.93:6831/264429558","heartbeat_front_addr":"172.21.15.93:6829/264429558","state":["exists","up"]},{"osd":11,"uuid":"6bbb5b79-8d4d-441f-9583-203df6c78cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6825","nonce":3389913754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6827","nonce":3389913754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6831","nonce":3389913754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6829","nonce":3389913754}]},"public_addr":"172.21.15.97:6825/3389913754","cluster_addr":"172.21.15.97:6827/3389913754","heartbeat_back_addr":"172.21.15.97:6831/3389913754","heartbeat_front_addr":"172.21.15.97:6829/3389913754","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:54:41.079618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:03.207899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:24.136961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:50.392609+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:56:12.136237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:56:33.212314+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:00.243646+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:22.059234+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:43.232399+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:10.043896+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:31.910780+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:53.158983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/2084033276":"2023-07-15T11:53:05.639597+0000","172.21.15.17:6801/1059284716":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/3651785636":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/1373538024":"2023-07-15T11:52:37.887946+0000","172.21.15.17:0/3560395020":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6800/1059284716":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/4262912861":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/2258375202":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6800/2804227742":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6801/2804227742":"2023-07-15T11:52:37.887946+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T11:59:51.155 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.174+0000 7f5c771ff700 1 -- 172.21.15.17:0/2243082430 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f5c600a1150 msgr2=0x7f5c600a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:51.155 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.174+0000 7f5c771ff700 1 --2- 172.21.15.17:0/2243082430 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f5c600a1150 0x7f5c600a3600 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5c64009c60 tx=0x7f5c64009260 comp rx=0 tx=0).stop 2023-07-14T11:59:51.155 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.174+0000 7f5c771ff700 1 -- 172.21.15.17:0/2243082430 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c70100e40 msgr2=0x7f5c70072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:51.155 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.174+0000 7f5c771ff700 1 --2- 172.21.15.17:0/2243082430 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c70100e40 0x7f5c70072900 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c00d980 tx=0x7f5c5c00dc90 comp rx=0 tx=0).stop 2023-07-14T11:59:51.155 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.174+0000 7f5c771ff700 1 -- 172.21.15.17:0/2243082430 shutdown_connections 2023-07-14T11:59:51.155 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.174+0000 7f5c771ff700 1 --2- 172.21.15.17:0/2243082430 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f5c600a1150 0x7f5c600a3600 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:51.156 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.174+0000 7f5c771ff700 1 --2- 172.21.15.17:0/2243082430 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c700ffbb0 0x7f5c700742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:51.156 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.174+0000 7f5c771ff700 1 --2- 172.21.15.17:0/2243082430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c701017f0 0x7f5c70072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:51.156 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.174+0000 7f5c771ff700 1 --2- 172.21.15.17:0/2243082430 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c70100e40 0x7f5c70072900 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:51.156 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.174+0000 7f5c771ff700 1 -- 172.21.15.17:0/2243082430 >> 172.21.15.17:0/2243082430 conn(0x7f5c700fb1d0 msgr2=0x7f5c701042d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:51.156 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.174+0000 7f5c771ff700 1 -- 172.21.15.17:0/2243082430 shutdown_connections 2023-07-14T11:59:51.156 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.174+0000 7f5c771ff700 1 -- 172.21.15.17:0/2243082430 wait complete. 2023-07-14T11:59:51.174 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-07-14T11:59:43.292866+0000', 'modified': '2023-07-14T11:59:48.798071+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24565}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24565': {'gid': 24565, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6841/3905201143', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6840', 'nonce': 3905201143}, {'type': 'v1', 'addr': '172.21.15.17:6841', 'nonce': 3905201143}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-07-14T11:59:51.175 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-07-14T11:59:51.175 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-07-14T11:59:51.380 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.399+0000 7f1e127ff700 1 -- 172.21.15.17:0/931471602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e0c100dd0 msgr2=0x7f1e0c101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:51.380 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.399+0000 7f1e127ff700 1 --2- 172.21.15.17:0/931471602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e0c100dd0 0x7f1e0c101220 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1e00009990 tx=0x7f1e00009ca0 comp rx=0 tx=0).stop 2023-07-14T11:59:51.380 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.399+0000 7f1e127ff700 1 -- 172.21.15.17:0/931471602 shutdown_connections 2023-07-14T11:59:51.380 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.399+0000 7f1e127ff700 1 --2- 172.21.15.17:0/931471602 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1e0c101760 0x7f1e0c108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:51.381 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.399+0000 7f1e127ff700 1 --2- 172.21.15.17:0/931471602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e0c0ffbe0 0x7f1e0c0ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:51.381 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.399+0000 7f1e127ff700 1 --2- 172.21.15.17:0/931471602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e0c100dd0 0x7f1e0c101220 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:51.381 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.399+0000 7f1e127ff700 1 -- 172.21.15.17:0/931471602 >> 172.21.15.17:0/931471602 conn(0x7f1e0c0fb200 msgr2=0x7f1e0c0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:51.381 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.399+0000 7f1e127ff700 1 -- 172.21.15.17:0/931471602 shutdown_connections 2023-07-14T11:59:51.381 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.399+0000 7f1e127ff700 1 -- 172.21.15.17:0/931471602 wait complete. 2023-07-14T11:59:51.383 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.400+0000 7f1e127ff700 1 Processor -- start 2023-07-14T11:59:51.383 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.400+0000 7f1e127ff700 1 -- start start 2023-07-14T11:59:51.383 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.401+0000 7f1e127ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e0c0ffbe0 0x7f1e0c197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:51.383 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.401+0000 7f1e127ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1e0c100dd0 0x7f1e0c198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:51.383 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.401+0000 7f1e127ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e0c101760 0x7f1e0c19f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:51.384 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.401+0000 7f1e127ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c19f6d0 con 0x7f1e0c0ffbe0 2023-07-14T11:59:51.384 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.401+0000 7f1e127ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c19f810 con 0x7f1e0c100dd0 2023-07-14T11:59:51.384 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.401+0000 7f1e127ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c19f9a0 con 0x7f1e0c101760 2023-07-14T11:59:51.384 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.401+0000 7f1e0bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e0c0ffbe0 0x7f1e0c197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:51.384 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.401+0000 7f1e0bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e0c0ffbe0 0x7f1e0c197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51966/0 (socket says 172.21.15.17:51966) 2023-07-14T11:59:51.384 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.401+0000 7f1e0bfff700 1 -- 172.21.15.17:0/786535743 learned_addr learned my addr 172.21.15.17:0/786535743 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:59:51.384 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.401+0000 7f1e0b7fe700 1 --2- 172.21.15.17:0/786535743 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1e0c100dd0 0x7f1e0c198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:51.385 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.401+0000 7f1e10d9c700 1 --2- 172.21.15.17:0/786535743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e0c101760 0x7f1e0c19f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:51.385 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.401+0000 7f1e0bfff700 1 -- 172.21.15.17:0/786535743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e0c101760 msgr2=0x7f1e0c19f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:51.385 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.401+0000 7f1e0bfff700 1 --2- 172.21.15.17:0/786535743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e0c101760 0x7f1e0c19f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:51.385 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.401+0000 7f1e0bfff700 1 -- 172.21.15.17:0/786535743 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1e0c100dd0 msgr2=0x7f1e0c198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:51.385 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.401+0000 7f1e0bfff700 1 --2- 172.21.15.17:0/786535743 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1e0c100dd0 0x7f1e0c198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:51.385 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.401+0000 7f1e0bfff700 1 -- 172.21.15.17:0/786535743 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dfc009520 con 0x7f1e0c0ffbe0 2023-07-14T11:59:51.385 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.401+0000 7f1e0bfff700 1 --2- 172.21.15.17:0/786535743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e0c0ffbe0 0x7f1e0c197b50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc00ea50 tx=0x7f1dfc00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:51.385 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.402+0000 7f1e097fa700 1 -- 172.21.15.17:0/786535743 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc00cbd0 con 0x7f1e0c0ffbe0 2023-07-14T11:59:51.386 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.402+0000 7f1e097fa700 1 -- 172.21.15.17:0/786535743 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1dfc00cd30 con 0x7f1e0c0ffbe0 2023-07-14T11:59:51.386 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.402+0000 7f1e097fa700 1 -- 172.21.15.17:0/786535743 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc018680 con 0x7f1e0c0ffbe0 2023-07-14T11:59:51.386 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.402+0000 7f1e127ff700 1 -- 172.21.15.17:0/786535743 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e000095f0 con 0x7f1e0c0ffbe0 2023-07-14T11:59:51.386 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.402+0000 7f1e127ff700 1 -- 172.21.15.17:0/786535743 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e0c19ffb0 con 0x7f1e0c0ffbe0 2023-07-14T11:59:51.386 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.405+0000 7f1e097fa700 1 -- 172.21.15.17:0/786535743 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1dfc010430 con 0x7f1e0c0ffbe0 2023-07-14T11:59:51.387 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.406+0000 7f1e2603fb80 1 Processor -- start 2023-07-14T11:59:51.387 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.407+0000 7f1e2603fb80 1 -- start start 2023-07-14T11:59:51.388 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.407+0000 7f1e097fa700 1 --2- 172.21.15.17:0/786535743 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f1df40a1150 0x7f1df40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:51.388 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.407+0000 7f1e097fa700 1 -- 172.21.15.17:0/786535743 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1dfc014070 con 0x7f1e0c0ffbe0 2023-07-14T11:59:51.388 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.407+0000 7f1e10d9c700 1 --2- 172.21.15.17:0/786535743 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f1df40a1150 0x7f1df40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:51.388 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.407+0000 7f1e2603fb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc0a37e480 0x55cc0a37e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:51.388 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.407+0000 7f1e2603fb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cc0a37edd0 0x55cc0a37f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:51.388 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.407+0000 7f1e2603fb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cc0a37fc20 0x55cc0a380090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:51.391 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.407+0000 7f1e2603fb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc0a283c60 con 0x55cc0a37edd0 2023-07-14T11:59:51.391 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.407+0000 7f1e2603fb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc0a2d69f0 con 0x55cc0a37fc20 2023-07-14T11:59:51.391 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.407+0000 7f1e2603fb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc0a2cfe80 con 0x55cc0a37e480 2023-07-14T11:59:51.391 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.408+0000 7f1e0b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cc0a37edd0 0x55cc0a37f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:51.391 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.408+0000 7f1e0b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cc0a37edd0 0x55cc0a37f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51968/0 (socket says 172.21.15.17:51968) 2023-07-14T11:59:51.391 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.408+0000 7f1e0b7fe700 1 -- 172.21.15.17:0/1921005894 learned_addr learned my addr 172.21.15.17:0/1921005894 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:59:51.392 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.408+0000 7f1e0bfff700 1 --2- 172.21.15.17:0/1921005894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc0a37e480 0x55cc0a37e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:51.392 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.408+0000 7f1e10d9c700 1 --2- 172.21.15.17:0/1921005894 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cc0a37fc20 0x55cc0a380090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:51.392 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.408+0000 7f1e0b7fe700 1 -- 172.21.15.17:0/1921005894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc0a37e480 msgr2=0x55cc0a37e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:59:51.392 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.408+0000 7f1e0b7fe700 1 --2- 172.21.15.17:0/1921005894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc0a37e480 0x55cc0a37e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:51.392 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.408+0000 7f1e0b7fe700 1 -- 172.21.15.17:0/1921005894 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cc0a37fc20 msgr2=0x55cc0a380090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:59:51.392 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.408+0000 7f1e0b7fe700 1 --2- 172.21.15.17:0/1921005894 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cc0a37fc20 0x55cc0a380090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:51.392 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.408+0000 7f1e10d9c700 1 --2- 172.21.15.17:0/786535743 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f1df40a1150 0x7f1df40a3600 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1e0400f540 tx=0x7f1e04009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:51.393 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.408+0000 7f1e0b7fe700 1 -- 172.21.15.17:0/1921005894 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc0a0fdd90 con 0x55cc0a37edd0 2023-07-14T11:59:51.393 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.408+0000 7f1e0bfff700 1 --2- 172.21.15.17:0/1921005894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc0a37e480 0x55cc0a37e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:59:51.393 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.408+0000 7f1e0b7fe700 1 --2- 172.21.15.17:0/1921005894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cc0a37edd0 0x55cc0a37f690 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1e00015000 tx=0x7f1e00005ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e81669ee0eb328ff server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:51.393 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.408+0000 7f1df27fc700 1 -- 172.21.15.17:0/1921005894 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0001d070 con 0x55cc0a37edd0 2023-07-14T11:59:51.393 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.409+0000 7f1df27fc700 1 -- 172.21.15.17:0/1921005894 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e0001e040 con 0x55cc0a37edd0 2023-07-14T11:59:51.393 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.409+0000 7f1df27fc700 1 -- 172.21.15.17:0/1921005894 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e000048d0 con 0x55cc0a37edd0 2023-07-14T11:59:51.393 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.409+0000 7f1e2603fb80 1 -- 172.21.15.17:0/1921005894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cc0a37edd0 msgr2=0x55cc0a37f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:51.393 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.409+0000 7f1e2603fb80 1 --2- 172.21.15.17:0/1921005894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cc0a37edd0 0x55cc0a37f690 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1e00015000 tx=0x7f1e00005ea0 comp rx=0 tx=0).stop 2023-07-14T11:59:51.394 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.409+0000 7f1e2603fb80 1 -- 172.21.15.17:0/1921005894 shutdown_connections 2023-07-14T11:59:51.394 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.409+0000 7f1e2603fb80 1 --2- 172.21.15.17:0/1921005894 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cc0a37fc20 0x55cc0a380090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:51.394 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.409+0000 7f1e2603fb80 1 --2- 172.21.15.17:0/1921005894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc0a37e480 0x55cc0a37e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:51.394 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.409+0000 7f1e2603fb80 1 --2- 172.21.15.17:0/1921005894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cc0a37edd0 0x55cc0a37f690 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:51.394 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.409+0000 7f1e2603fb80 1 -- 172.21.15.17:0/1921005894 >> 172.21.15.17:0/1921005894 conn(0x55cc0a373d50 msgr2=0x55cc0a379180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:51.394 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.409+0000 7f1e2603fb80 1 -- 172.21.15.17:0/1921005894 shutdown_connections 2023-07-14T11:59:51.394 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.410+0000 7f1e2603fb80 1 -- 172.21.15.17:0/1921005894 wait complete. 2023-07-14T11:59:51.394 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.410+0000 7f1e2603fb80 1 -- start start 2023-07-14T11:59:51.395 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.410+0000 7f1e2603fb80 1 Processor -- start 2023-07-14T11:59:51.397 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.411+0000 7f1e2603fb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cc0a37e480 0x55cc0a3880f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:51.397 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.411+0000 7f1e2603fb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc0a37edd0 0x55cc0a388630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:51.398 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.411+0000 7f1e2603fb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cc0a37fc20 0x55cc0a38f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:51.398 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.411+0000 7f1e2603fb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc0a26fa70 con 0x55cc0a37e480 2023-07-14T11:59:51.399 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.411+0000 7f1e2603fb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc0a26f770 con 0x55cc0a37fc20 2023-07-14T11:59:51.399 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.411+0000 7f1e2603fb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc0a1538d0 con 0x55cc0a37edd0 2023-07-14T11:59:51.399 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.411+0000 7f1e2603fb80 10 client.0 _ll_register_callbacks cb 0x55cc0a26d710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:59:51.400 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.411+0000 7f1e0bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cc0a37e480 0x55cc0a3880f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:51.400 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.411+0000 7f1e0bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cc0a37e480 0x55cc0a3880f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51978/0 (socket says 172.21.15.17:51978) 2023-07-14T11:59:51.400 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.411+0000 7f1e0bfff700 1 -- 172.21.15.17:0/1199476262 learned_addr learned my addr 172.21.15.17:0/1199476262 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:59:51.401 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.411+0000 7f1e0b7fe700 1 --2- 172.21.15.17:0/1199476262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc0a37edd0 0x55cc0a388630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:51.401 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.411+0000 7f1e10d9c700 1 --2- 172.21.15.17:0/1199476262 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cc0a37fc20 0x55cc0a38f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:51.401 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.411+0000 7f1e0bfff700 1 -- 172.21.15.17:0/1199476262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc0a37edd0 msgr2=0x55cc0a388630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:59:51.401 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.411+0000 7f1e0bfff700 1 --2- 172.21.15.17:0/1199476262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc0a37edd0 0x55cc0a388630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:51.402 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.411+0000 7f1e0bfff700 1 -- 172.21.15.17:0/1199476262 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cc0a37fc20 msgr2=0x55cc0a38f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:59:51.402 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.411+0000 7f1e0bfff700 1 --2- 172.21.15.17:0/1199476262 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cc0a37fc20 0x55cc0a38f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:51.402 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.411+0000 7f1e0bfff700 1 -- 172.21.15.17:0/1199476262 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e0c19ffb0 con 0x55cc0a37e480 2023-07-14T11:59:51.402 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.412+0000 7f1e0bfff700 1 --2- 172.21.15.17:0/1199476262 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cc0a37e480 0x55cc0a3880f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc00c5b0 tx=0x7f1dfc09a700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed8b696c01428304 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:51.403 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.412+0000 7f1deb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-07-14T11:59:51.403 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.412+0000 7f1deb7fe700 1 -- 172.21.15.17:0/1199476262 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc0a2380 con 0x55cc0a37e480 2023-07-14T11:59:51.403 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.412+0000 7f1deb7fe700 1 -- 172.21.15.17:0/1199476262 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1dfc0a2610 con 0x55cc0a37e480 2023-07-14T11:59:51.404 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.412+0000 7f1e2603fb80 1 -- 172.21.15.17:0/1199476262 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cc0a2294a0 con 0x55cc0a37e480 2023-07-14T11:59:51.404 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.412+0000 7f1deb7fe700 1 -- 172.21.15.17:0/1199476262 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1dfc0106e0 con 0x55cc0a37e480 2023-07-14T11:59:51.404 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.412+0000 7f1e2603fb80 10 client.14661 fetch_fsmap learned FSMap version 17 2023-07-14T11:59:51.405 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.412+0000 7f1e2603fb80 1 -- 172.21.15.17:0/1199476262 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x7f1e0c108950 con 0x55cc0a37e480 2023-07-14T11:59:51.405 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.412+0000 7f1deb7fe700 1 -- 172.21.15.17:0/1199476262 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1dfc0c8080 con 0x55cc0a37e480 2023-07-14T11:59:51.405 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.412+0000 7f1deb7fe700 1 -- 172.21.15.17:0/1199476262 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 17) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f1dfc099120 con 0x55cc0a37e480 2023-07-14T11:59:51.405 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.413+0000 7f1e2603fb80 10 client.14661 fetch_fsmap finished waiting for FSMap version 17 2023-07-14T11:59:51.405 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.413+0000 7f1e2603fb80 10 client.14661 resolve_mds: resolved name 'a' to daemon mds.a 2023-07-14T11:59:51.405 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.413+0000 7f1e2603fb80 20 client.14661 populate_metadata read hostname 'smithi017' 2023-07-14T11:59:51.406 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.413+0000 7f1e2603fb80 1 --2- 172.21.15.17:0/1199476262 >> [v2:172.21.15.17:6840/3905201143,v1:172.21.15.17:6841/3905201143] conn(0x55cc0a393200 0x55cc0a3926c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:51.406 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.413+0000 7f1e2603fb80 4 client.14661 mds_command: new command op to 24565 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T11:59:51.406 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.413+0000 7f1e2603fb80 1 -- 172.21.15.17:0/1199476262 --> [v2:172.21.15.17:6840/3905201143,v1:172.21.15.17:6841/3905201143] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cc0a2c8e60 con 0x55cc0a393200 2023-07-14T11:59:51.406 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.413+0000 7f1e0b7fe700 1 --2- 172.21.15.17:0/1199476262 >> [v2:172.21.15.17:6840/3905201143,v1:172.21.15.17:6841/3905201143] conn(0x55cc0a393200 0x55cc0a3926c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:51.406 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.413+0000 7f1e0b7fe700 1 --2- 172.21.15.17:0/1199476262 >> [v2:172.21.15.17:6840/3905201143,v1:172.21.15.17:6841/3905201143] conn(0x55cc0a393200 0x55cc0a3926c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4156887f2addbb54 server_cookie=963f8dd4f3f5643b in_seq=0 out_seq=0 2023-07-14T11:59:51.406 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.413+0000 7f1deb7fe700 10 client.14661 ms_handle_connect on v2:172.21.15.17:6840/3905201143 2023-07-14T11:59:51.406 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.414+0000 7f1deb7fe700 1 -- 172.21.15.17:0/1199476262 <== mds.0 v2:172.21.15.17:6840/3905201143 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cc0a2c8e60 con 0x55cc0a393200 2023-07-14T11:59:51.406 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.414+0000 7f1deb7fe700 10 client.14661 handle_command_reply: tid=0 2023-07-14T11:59:51.407 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.414+0000 7f1e2603fb80 1 client.14661 shutdown 2023-07-14T11:59:51.407 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.414+0000 7f1e2603fb80 1 -- 172.21.15.17:0/1199476262 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cc0a37e480 msgr2=0x55cc0a3880f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:51.407 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.414+0000 7f1e2603fb80 1 --2- 172.21.15.17:0/1199476262 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cc0a37e480 0x55cc0a3880f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc00c5b0 tx=0x7f1dfc09a700 comp rx=0 tx=0).stop 2023-07-14T11:59:51.407 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.414+0000 7f1e2603fb80 1 -- 172.21.15.17:0/1199476262 shutdown_connections 2023-07-14T11:59:51.407 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.414+0000 7f1e2603fb80 1 --2- 172.21.15.17:0/1199476262 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cc0a37fc20 0x55cc0a38f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:51.407 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.414+0000 7f1e2603fb80 1 --2- 172.21.15.17:0/1199476262 >> [v2:172.21.15.17:6840/3905201143,v1:172.21.15.17:6841/3905201143] conn(0x55cc0a393200 0x55cc0a3926c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:51.407 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.414+0000 7f1e2603fb80 1 --2- 172.21.15.17:0/1199476262 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cc0a37e480 0x55cc0a3880f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:51.408 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.414+0000 7f1e2603fb80 1 --2- 172.21.15.17:0/1199476262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc0a37edd0 0x55cc0a388630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:51.408 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.414+0000 7f1deb7fe700 0 client.14661 ms_handle_reset on v2:172.21.15.17:6840/3905201143 2023-07-14T11:59:51.408 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.414+0000 7f1e2603fb80 1 -- 172.21.15.17:0/1199476262 >> 172.21.15.17:0/1199476262 conn(0x55cc0a373d50 msgr2=0x55cc0a378860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:51.408 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.415+0000 7f1e2603fb80 1 -- 172.21.15.17:0/1199476262 shutdown_connections 2023-07-14T11:59:51.408 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.415+0000 7f1e2603fb80 1 -- 172.21.15.17:0/1199476262 wait complete. 2023-07-14T11:59:51.408 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.415+0000 7f1e2603fb80 20 client.14661 trim_cache size 0 max 16384 2023-07-14T11:59:51.414 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.433+0000 7f1e2603fb80 1 Processor -- start 2023-07-14T11:59:51.414 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.433+0000 7f1e2603fb80 1 -- start start 2023-07-14T11:59:51.414 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.433+0000 7f1e2603fb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cc0a37e480 0x55cc0a386dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:51.415 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.433+0000 7f1e2603fb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cc0a37edd0 0x55cc0a38f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:51.415 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.433+0000 7f1e2603fb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc0a37fc20 0x55cc0a3926c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:51.415 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.433+0000 7f1e2603fb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc0a15fe10 con 0x55cc0a37e480 2023-07-14T11:59:51.415 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.433+0000 7f1e2603fb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc0a28e4b0 con 0x55cc0a37edd0 2023-07-14T11:59:51.415 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.433+0000 7f1e2603fb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc0a3944b0 con 0x55cc0a37fc20 2023-07-14T11:59:51.415 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.434+0000 7f1e0bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cc0a37e480 0x55cc0a386dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:51.415 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.434+0000 7f1e0bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cc0a37e480 0x55cc0a386dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51994/0 (socket says 172.21.15.17:51994) 2023-07-14T11:59:51.416 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.434+0000 7f1e0bfff700 1 -- 172.21.15.17:0/2122731688 learned_addr learned my addr 172.21.15.17:0/2122731688 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:59:51.416 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.434+0000 7f1e0b7fe700 1 --2- 172.21.15.17:0/2122731688 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cc0a37edd0 0x55cc0a38f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:51.416 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.434+0000 7f1e10d9c700 1 --2- 172.21.15.17:0/2122731688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc0a37fc20 0x55cc0a3926c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:51.419 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.434+0000 7f1e0bfff700 1 -- 172.21.15.17:0/2122731688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc0a37fc20 msgr2=0x55cc0a3926c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:59:51.419 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.434+0000 7f1e0bfff700 1 --2- 172.21.15.17:0/2122731688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc0a37fc20 0x55cc0a3926c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:51.419 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.434+0000 7f1e0bfff700 1 -- 172.21.15.17:0/2122731688 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cc0a37edd0 msgr2=0x55cc0a38f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:59:51.419 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.434+0000 7f1e0bfff700 1 --2- 172.21.15.17:0/2122731688 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cc0a37edd0 0x55cc0a38f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:51.419 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.434+0000 7f1e0bfff700 1 -- 172.21.15.17:0/2122731688 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc0a0e6c10 con 0x55cc0a37e480 2023-07-14T11:59:51.419 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.434+0000 7f1e0b7fe700 1 --2- 172.21.15.17:0/2122731688 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cc0a37edd0 0x55cc0a38f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:59:51.420 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.434+0000 7f1e0bfff700 1 --2- 172.21.15.17:0/2122731688 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cc0a37e480 0x55cc0a386dd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc096c30 tx=0x7f1dfc098770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3613f457d7f8f5e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:51.420 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.434+0000 7f1df2ffd700 1 -- 172.21.15.17:0/2122731688 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc0a2610 con 0x55cc0a37e480 2023-07-14T11:59:51.420 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.435+0000 7f1df2ffd700 1 -- 172.21.15.17:0/2122731688 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cc0a2294a0 con 0x55cc0a37e480 2023-07-14T11:59:51.420 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.435+0000 7f1df2ffd700 1 -- 172.21.15.17:0/2122731688 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc0106e0 con 0x55cc0a37e480 2023-07-14T11:59:51.420 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.435+0000 7f1e2603fb80 1 -- 172.21.15.17:0/2122731688 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cc0a37e480 msgr2=0x55cc0a386dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:51.420 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.435+0000 7f1e2603fb80 1 --2- 172.21.15.17:0/2122731688 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cc0a37e480 0x55cc0a386dd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc096c30 tx=0x7f1dfc098770 comp rx=0 tx=0).stop 2023-07-14T11:59:51.420 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.435+0000 7f1e2603fb80 1 -- 172.21.15.17:0/2122731688 shutdown_connections 2023-07-14T11:59:51.421 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.435+0000 7f1e2603fb80 1 --2- 172.21.15.17:0/2122731688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc0a37fc20 0x55cc0a3926c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:51.421 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.435+0000 7f1e2603fb80 1 --2- 172.21.15.17:0/2122731688 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cc0a37e480 0x55cc0a386dd0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:51.421 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.435+0000 7f1e2603fb80 1 --2- 172.21.15.17:0/2122731688 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cc0a37edd0 0x55cc0a38f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:51.421 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.435+0000 7f1e2603fb80 1 -- 172.21.15.17:0/2122731688 >> 172.21.15.17:0/2122731688 conn(0x55cc0a373d50 msgr2=0x55cc0a39bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:51.421 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.435+0000 7f1e2603fb80 1 -- 172.21.15.17:0/2122731688 shutdown_connections 2023-07-14T11:59:51.421 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.435+0000 7f1e2603fb80 1 -- 172.21.15.17:0/2122731688 wait complete. 2023-07-14T11:59:51.421 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.436+0000 7f1e2603fb80 1 -- start start 2023-07-14T11:59:51.421 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.436+0000 7f1e2603fb80 1 Processor -- start 2023-07-14T11:59:51.422 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.436+0000 7f1e2603fb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cc0a37e480 0x55cc0a38a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:51.422 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.436+0000 7f1e2603fb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc0a37edd0 0x55cc0a38f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:51.422 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.436+0000 7f1e2603fb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cc0a37fc20 0x55cc0a38af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:51.422 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.436+0000 7f1e2603fb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc0a3e70b0 con 0x55cc0a37e480 2023-07-14T11:59:51.422 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.436+0000 7f1e2603fb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc0a3e7220 con 0x55cc0a37fc20 2023-07-14T11:59:51.422 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.436+0000 7f1e2603fb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc0a3e73e0 con 0x55cc0a37edd0 2023-07-14T11:59:51.422 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.436+0000 7f1e2603fb80 10 client.0 _ll_register_callbacks cb 0x55cc0a26d710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T11:59:51.423 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.436+0000 7f1e0bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cc0a37e480 0x55cc0a38a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:51.423 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.436+0000 7f1e0bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cc0a37e480 0x55cc0a38a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52010/0 (socket says 172.21.15.17:52010) 2023-07-14T11:59:51.423 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.436+0000 7f1e0bfff700 1 -- 172.21.15.17:0/3197159756 learned_addr learned my addr 172.21.15.17:0/3197159756 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:59:51.423 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.437+0000 7f1e10d9c700 1 --2- 172.21.15.17:0/3197159756 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cc0a37fc20 0x55cc0a38af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:51.423 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.437+0000 7f1e0b7fe700 1 --2- 172.21.15.17:0/3197159756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc0a37edd0 0x55cc0a38f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:51.423 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.437+0000 7f1e0bfff700 1 -- 172.21.15.17:0/3197159756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc0a37edd0 msgr2=0x55cc0a38f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:59:51.423 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.437+0000 7f1e0bfff700 1 --2- 172.21.15.17:0/3197159756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc0a37edd0 0x55cc0a38f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:51.423 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.437+0000 7f1e0bfff700 1 -- 172.21.15.17:0/3197159756 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cc0a37fc20 msgr2=0x55cc0a38af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T11:59:51.424 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.437+0000 7f1e0bfff700 1 --2- 172.21.15.17:0/3197159756 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cc0a37fc20 0x55cc0a38af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:51.424 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.437+0000 7f1e0bfff700 1 -- 172.21.15.17:0/3197159756 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cc0a0fdd90 con 0x55cc0a37e480 2023-07-14T11:59:51.424 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.437+0000 7f1e0b7fe700 1 --2- 172.21.15.17:0/3197159756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc0a37edd0 0x55cc0a38f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:59:51.424 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.437+0000 7f1e0bfff700 1 --2- 172.21.15.17:0/3197159756 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cc0a37e480 0x55cc0a38a8a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc083a10 tx=0x7f1dfc09e330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25f8faf162659ac server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:51.424 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.437+0000 7f1deb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-07-14T11:59:51.424 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.437+0000 7f1deb7fe700 1 -- 172.21.15.17:0/3197159756 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc0a2610 con 0x55cc0a37e480 2023-07-14T11:59:51.424 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.437+0000 7f1deb7fe700 1 -- 172.21.15.17:0/3197159756 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1dfc0b2ba0 con 0x55cc0a37e480 2023-07-14T11:59:51.425 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.437+0000 7f1e2603fb80 1 -- 172.21.15.17:0/3197159756 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1e0001e040 con 0x55cc0a37e480 2023-07-14T11:59:51.425 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.437+0000 7f1deb7fe700 1 -- 172.21.15.17:0/3197159756 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1dfc0b2d60 con 0x55cc0a37e480 2023-07-14T11:59:51.425 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.437+0000 7f1e2603fb80 10 client.14667 fetch_fsmap learned FSMap version 17 2023-07-14T11:59:51.425 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.437+0000 7f1e2603fb80 1 -- 172.21.15.17:0/3197159756 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x7f1e0c108950 con 0x55cc0a37e480 2023-07-14T11:59:51.425 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.438+0000 7f1deb7fe700 1 -- 172.21.15.17:0/3197159756 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e0001e040 con 0x55cc0a37e480 2023-07-14T11:59:51.425 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.438+0000 7f1deb7fe700 1 -- 172.21.15.17:0/3197159756 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 17) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f1dfc07e070 con 0x55cc0a37e480 2023-07-14T11:59:51.425 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.438+0000 7f1e2603fb80 10 client.14667 fetch_fsmap finished waiting for FSMap version 17 2023-07-14T11:59:51.425 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.438+0000 7f1e2603fb80 10 client.14667 resolve_mds: resolved name 'a' to daemon mds.a 2023-07-14T11:59:51.426 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.438+0000 7f1e2603fb80 20 client.14667 populate_metadata read hostname 'smithi017' 2023-07-14T11:59:51.426 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.438+0000 7f1e2603fb80 1 --2- 172.21.15.17:0/3197159756 >> [v2:172.21.15.17:6840/3905201143,v1:172.21.15.17:6841/3905201143] conn(0x55cc0a3ebc00 0x55cc0a3e8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:51.426 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.438+0000 7f1e2603fb80 4 client.14667 mds_command: new command op to 24565 tid=0[{"prefix": "status"}] 2023-07-14T11:59:51.426 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.438+0000 7f1e2603fb80 1 -- 172.21.15.17:0/3197159756 --> [v2:172.21.15.17:6840/3905201143,v1:172.21.15.17:6841/3905201143] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55cc0a28b000 con 0x55cc0a3ebc00 2023-07-14T11:59:51.426 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.438+0000 7f1e0b7fe700 1 --2- 172.21.15.17:0/3197159756 >> [v2:172.21.15.17:6840/3905201143,v1:172.21.15.17:6841/3905201143] conn(0x55cc0a3ebc00 0x55cc0a3e8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:51.427 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.439+0000 7f1e0b7fe700 1 --2- 172.21.15.17:0/3197159756 >> [v2:172.21.15.17:6840/3905201143,v1:172.21.15.17:6841/3905201143] conn(0x55cc0a3ebc00 0x55cc0a3e8590 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=592cec5d43f18d99 server_cookie=167776a820409d42 in_seq=0 out_seq=0 2023-07-14T11:59:51.427 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.439+0000 7f1deb7fe700 10 client.14667 ms_handle_connect on v2:172.21.15.17:6840/3905201143 2023-07-14T11:59:51.427 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.439+0000 7f1deb7fe700 1 -- 172.21.15.17:0/3197159756 <== mds.0 v2:172.21.15.17:6840/3905201143 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55cc0a28b000 con 0x55cc0a3ebc00 2023-07-14T11:59:51.428 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.439+0000 7f1deb7fe700 10 client.14667 handle_command_reply: tid=0 2023-07-14T11:59:51.428 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.439+0000 7f1e2603fb80 1 client.14667 shutdown 2023-07-14T11:59:51.428 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.439+0000 7f1e2603fb80 1 -- 172.21.15.17:0/3197159756 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cc0a37e480 msgr2=0x55cc0a38a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:51.428 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.439+0000 7f1e2603fb80 1 --2- 172.21.15.17:0/3197159756 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cc0a37e480 0x55cc0a38a8a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc083a10 tx=0x7f1dfc09e330 comp rx=0 tx=0).stop 2023-07-14T11:59:51.428 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.439+0000 7f1e2603fb80 1 -- 172.21.15.17:0/3197159756 shutdown_connections 2023-07-14T11:59:51.428 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.439+0000 7f1e2603fb80 1 --2- 172.21.15.17:0/3197159756 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cc0a37fc20 0x55cc0a38af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:51.428 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.439+0000 7f1e2603fb80 1 --2- 172.21.15.17:0/3197159756 >> [v2:172.21.15.17:6840/3905201143,v1:172.21.15.17:6841/3905201143] conn(0x55cc0a3ebc00 0x55cc0a3e8590 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:51.429 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.439+0000 7f1deb7fe700 0 client.14667 ms_handle_reset on v2:172.21.15.17:6840/3905201143 2023-07-14T11:59:51.429 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.439+0000 7f1e2603fb80 1 --2- 172.21.15.17:0/3197159756 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cc0a37e480 0x55cc0a38a8a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:51.429 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.439+0000 7f1e2603fb80 1 --2- 172.21.15.17:0/3197159756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc0a37edd0 0x55cc0a38f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:51.429 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.439+0000 7f1e2603fb80 1 -- 172.21.15.17:0/3197159756 >> 172.21.15.17:0/3197159756 conn(0x55cc0a373d50 msgr2=0x55cc0a386dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:51.429 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.439+0000 7f1e2603fb80 1 -- 172.21.15.17:0/3197159756 shutdown_connections 2023-07-14T11:59:51.429 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.439+0000 7f1e2603fb80 1 -- 172.21.15.17:0/3197159756 wait complete. 2023-07-14T11:59:51.429 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.440+0000 7f1e2603fb80 20 client.14667 trim_cache size 0 max 16384 2023-07-14T11:59:51.430 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2023-07-14T11:59:51.430 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "b3c386cc-223c-11ee-9b33-001a4aab830c", 2023-07-14T11:59:51.430 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2023-07-14T11:59:51.430 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 24565, 2023-07-14T11:59:51.430 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2023-07-14T11:59:51.430 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2023-07-14T11:59:51.431 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2023-07-14T11:59:51.431 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 7.6981412520000001, 2023-07-14T11:59:51.431 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 17, 2023-07-14T11:59:51.431 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 88, 2023-07-14T11:59:51.431 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 88, 2023-07-14T11:59:51.431 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 16.253222620999999 2023-07-14T11:59:51.431 INFO:teuthology.orchestra.run.smithi017.stdout:} 2023-07-14T11:59:51.432 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.440+0000 7f1e127ff700 1 -- 172.21.15.17:0/786535743 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f1df40a1150 msgr2=0x7f1df40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:51.432 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.440+0000 7f1e127ff700 1 --2- 172.21.15.17:0/786535743 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f1df40a1150 0x7f1df40a3600 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1e0400f540 tx=0x7f1e04009920 comp rx=0 tx=0).stop 2023-07-14T11:59:51.432 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.440+0000 7f1e127ff700 1 -- 172.21.15.17:0/786535743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e0c0ffbe0 msgr2=0x7f1e0c197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:51.432 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.440+0000 7f1e127ff700 1 --2- 172.21.15.17:0/786535743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e0c0ffbe0 0x7f1e0c197b50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc00ea50 tx=0x7f1dfc00ed60 comp rx=0 tx=0).stop 2023-07-14T11:59:51.433 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.440+0000 7f1e127ff700 1 -- 172.21.15.17:0/786535743 shutdown_connections 2023-07-14T11:59:51.433 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.440+0000 7f1e127ff700 1 --2- 172.21.15.17:0/786535743 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f1df40a1150 0x7f1df40a3600 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:51.433 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.440+0000 7f1e127ff700 1 --2- 172.21.15.17:0/786535743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e0c101760 0x7f1e0c19f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:51.433 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.440+0000 7f1e127ff700 1 --2- 172.21.15.17:0/786535743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e0c0ffbe0 0x7f1e0c197b50 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:51.433 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.440+0000 7f1e127ff700 1 --2- 172.21.15.17:0/786535743 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1e0c100dd0 0x7f1e0c198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:51.433 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.440+0000 7f1e127ff700 1 -- 172.21.15.17:0/786535743 >> 172.21.15.17:0/786535743 conn(0x7f1e0c0fb200 msgr2=0x7f1e0c1066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:51.433 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.440+0000 7f1e127ff700 1 -- 172.21.15.17:0/786535743 shutdown_connections 2023-07-14T11:59:51.433 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:51.440+0000 7f1e127ff700 1 -- 172.21.15.17:0/786535743 wait complete. 2023-07-14T11:59:51.439 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-14T11:59:51.440 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-07-14T11:59:51.441 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph fs ls 2023-07-14T11:59:52.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:51 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/2243082430' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:59:52.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:51 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/2243082430' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:59:52.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:51 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/2243082430' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T11:59:52.168 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T11:59:53.055 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:52 smithi017 ceph-mon[126339]: pgmap v281: 129 pgs: 11 creating+peering, 118 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2023-07-14T11:59:53.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:52 smithi093 ceph-mon[128922]: pgmap v281: 129 pgs: 11 creating+peering, 118 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2023-07-14T11:59:53.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:52 smithi097 ceph-mon[129313]: pgmap v281: 129 pgs: 11 creating+peering, 118 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2023-07-14T11:59:53.675 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.694+0000 7fb908196700 1 -- 172.21.15.17:0/994034700 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb9001007a0 msgr2=0x7fb900104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:53.676 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.694+0000 7fb908196700 1 --2- 172.21.15.17:0/994034700 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb9001007a0 0x7fb900104770 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0009990 tx=0x7fb8f0009ca0 comp rx=0 tx=0).stop 2023-07-14T11:59:53.676 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.694+0000 7fb908196700 1 -- 172.21.15.17:0/994034700 shutdown_connections 2023-07-14T11:59:53.676 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.694+0000 7fb908196700 1 --2- 172.21.15.17:0/994034700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb900104ea0 0x7fb90010ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:53.677 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.694+0000 7fb908196700 1 --2- 172.21.15.17:0/994034700 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9000ffe00 0x7fb9001001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:53.677 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.694+0000 7fb908196700 1 --2- 172.21.15.17:0/994034700 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb9001007a0 0x7fb900104770 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:53.677 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.694+0000 7fb908196700 1 -- 172.21.15.17:0/994034700 >> 172.21.15.17:0/994034700 conn(0x7fb9000fb660 msgr2=0x7fb9000fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:53.677 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.694+0000 7fb908196700 1 -- 172.21.15.17:0/994034700 shutdown_connections 2023-07-14T11:59:53.678 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.694+0000 7fb908196700 1 -- 172.21.15.17:0/994034700 wait complete. 2023-07-14T11:59:53.680 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.695+0000 7fb908196700 1 Processor -- start 2023-07-14T11:59:53.680 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.695+0000 7fb908196700 1 -- start start 2023-07-14T11:59:53.680 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.696+0000 7fb908196700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9000ffe00 0x7fb900072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:53.680 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.696+0000 7fb908196700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9001007a0 0x7fb900072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:53.681 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.696+0000 7fb908196700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb900104ea0 0x7fb900073290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:53.681 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.696+0000 7fb908196700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9001a4110 con 0x7fb9001007a0 2023-07-14T11:59:53.681 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.696+0000 7fb908196700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9001a4280 con 0x7fb900104ea0 2023-07-14T11:59:53.681 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.696+0000 7fb908196700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9001a4440 con 0x7fb9000ffe00 2023-07-14T11:59:53.681 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.696+0000 7fb905731700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9001007a0 0x7fb900072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:53.681 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.696+0000 7fb905731700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9001007a0 0x7fb900072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34606/0 (socket says 172.21.15.17:34606) 2023-07-14T11:59:53.681 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.696+0000 7fb905731700 1 -- 172.21.15.17:0/4222157508 learned_addr learned my addr 172.21.15.17:0/4222157508 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:59:53.682 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.696+0000 7fb905f32700 1 --2- 172.21.15.17:0/4222157508 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9000ffe00 0x7fb900072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:53.682 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.697+0000 7fb906733700 1 --2- 172.21.15.17:0/4222157508 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb900104ea0 0x7fb900073290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:53.682 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.697+0000 7fb905731700 1 -- 172.21.15.17:0/4222157508 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9000ffe00 msgr2=0x7fb900072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:53.682 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.697+0000 7fb905731700 1 --2- 172.21.15.17:0/4222157508 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9000ffe00 0x7fb900072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:53.682 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.697+0000 7fb905731700 1 -- 172.21.15.17:0/4222157508 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb900104ea0 msgr2=0x7fb900073290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:53.683 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.697+0000 7fb905731700 1 --2- 172.21.15.17:0/4222157508 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb900104ea0 0x7fb900073290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:53.683 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.697+0000 7fb905731700 1 -- 172.21.15.17:0/4222157508 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f00095f0 con 0x7fb9001007a0 2023-07-14T11:59:53.683 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.697+0000 7fb905f32700 1 --2- 172.21.15.17:0/4222157508 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9000ffe00 0x7fb900072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T11:59:53.683 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.697+0000 7fb906733700 1 --2- 172.21.15.17:0/4222157508 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb900104ea0 0x7fb900073290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T11:59:53.683 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.697+0000 7fb905731700 1 --2- 172.21.15.17:0/4222157508 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9001007a0 0x7fb900072d50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0000c00 tx=0x7fb8f0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:53.683 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.697+0000 7fb8f6ffd700 1 -- 172.21.15.17:0/4222157508 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f001d070 con 0x7fb9001007a0 2023-07-14T11:59:53.683 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.697+0000 7fb908196700 1 -- 172.21.15.17:0/4222157508 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9001a46c0 con 0x7fb9001007a0 2023-07-14T11:59:53.683 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.697+0000 7fb8f6ffd700 1 -- 172.21.15.17:0/4222157508 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb8f0021410 con 0x7fb9001007a0 2023-07-14T11:59:53.684 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.697+0000 7fb8f6ffd700 1 -- 172.21.15.17:0/4222157508 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f0017650 con 0x7fb9001007a0 2023-07-14T11:59:53.684 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.698+0000 7fb908196700 1 -- 172.21.15.17:0/4222157508 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9001a4b30 con 0x7fb9001007a0 2023-07-14T11:59:53.684 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.699+0000 7fb908196700 1 -- 172.21.15.17:0/4222157508 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb90004e210 con 0x7fb9001007a0 2023-07-14T11:59:53.685 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.700+0000 7fb8f6ffd700 1 -- 172.21.15.17:0/4222157508 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb8f0021580 con 0x7fb9001007a0 2023-07-14T11:59:53.685 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.702+0000 7fb8f6ffd700 1 --2- 172.21.15.17:0/4222157508 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fb8ec0a1150 0x7fb8ec0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:53.685 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.702+0000 7fb8f6ffd700 1 -- 172.21.15.17:0/4222157508 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fb8f00cdce0 con 0x7fb9001007a0 2023-07-14T11:59:53.685 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.702+0000 7fb905f32700 1 --2- 172.21.15.17:0/4222157508 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fb8ec0a1150 0x7fb8ec0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:53.686 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.703+0000 7fb905f32700 1 --2- 172.21.15.17:0/4222157508 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fb8ec0a1150 0x7fb8ec0a3600 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc011490 tx=0x7fb8fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:53.686 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.706+0000 7fb8f6ffd700 1 -- 172.21.15.17:0/4222157508 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb8f0097270 con 0x7fb9001007a0 2023-07-14T11:59:53.937 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.956+0000 7fb908196700 1 -- 172.21.15.17:0/4222157508 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb9001a4d50 con 0x7fb9001007a0 2023-07-14T11:59:53.937 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.957+0000 7fb8f6ffd700 1 -- 172.21.15.17:0/4222157508 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7fb8f009a890 con 0x7fb9001007a0 2023-07-14T11:59:53.938 INFO:teuthology.orchestra.run.smithi017.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-07-14T11:59:53.940 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.959+0000 7fb908196700 1 -- 172.21.15.17:0/4222157508 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fb8ec0a1150 msgr2=0x7fb8ec0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:53.940 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.959+0000 7fb908196700 1 --2- 172.21.15.17:0/4222157508 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fb8ec0a1150 0x7fb8ec0a3600 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc011490 tx=0x7fb8fc009260 comp rx=0 tx=0).stop 2023-07-14T11:59:53.941 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.959+0000 7fb908196700 1 -- 172.21.15.17:0/4222157508 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9001007a0 msgr2=0x7fb900072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:53.941 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.959+0000 7fb908196700 1 --2- 172.21.15.17:0/4222157508 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9001007a0 0x7fb900072d50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0000c00 tx=0x7fb8f0005740 comp rx=0 tx=0).stop 2023-07-14T11:59:53.941 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.959+0000 7fb908196700 1 -- 172.21.15.17:0/4222157508 shutdown_connections 2023-07-14T11:59:53.942 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.959+0000 7fb908196700 1 --2- 172.21.15.17:0/4222157508 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fb8ec0a1150 0x7fb8ec0a3600 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:53.942 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.959+0000 7fb908196700 1 --2- 172.21.15.17:0/4222157508 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb900104ea0 0x7fb900073290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:53.942 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.959+0000 7fb908196700 1 --2- 172.21.15.17:0/4222157508 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9000ffe00 0x7fb900072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:53.942 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.959+0000 7fb908196700 1 --2- 172.21.15.17:0/4222157508 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9001007a0 0x7fb900072d50 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:53.942 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.959+0000 7fb908196700 1 -- 172.21.15.17:0/4222157508 >> 172.21.15.17:0/4222157508 conn(0x7fb9000fb660 msgr2=0x7fb9000fc100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:53.942 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.960+0000 7fb908196700 1 -- 172.21.15.17:0/4222157508 shutdown_connections 2023-07-14T11:59:53.943 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:53.960+0000 7fb908196700 1 -- 172.21.15.17:0/4222157508 wait complete. 2023-07-14T11:59:54.549 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-07-14T11:59:54.550 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-14T11:59:54.550 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi017.front.sepia.ceph.com 2023-07-14T11:59:54.550 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-07-14T11:59:54.550 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-07-14T11:59:54.551 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-07-14T11:59:54.551 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-14T11:59:54.551 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-14T11:59:54.551 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-07-14T11:59:54.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:59:54.552 DEBUG:teuthology.orchestra.run.smithi017:> ip addr 2023-07-14T11:59:54.598 INFO:teuthology.orchestra.run.smithi017.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-14T11:59:54.598 INFO:teuthology.orchestra.run.smithi017.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-14T11:59:54.598 INFO:teuthology.orchestra.run.smithi017.stdout: inet 127.0.0.1/8 scope host lo 2023-07-14T11:59:54.598 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:59:54.599 INFO:teuthology.orchestra.run.smithi017.stdout: inet6 ::1/128 scope host 2023-07-14T11:59:54.599 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:59:54.599 INFO:teuthology.orchestra.run.smithi017.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-14T11:59:54.599 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 0c:c4:7a:6b:fe:00 brd ff:ff:ff:ff:ff:ff 2023-07-14T11:59:54.599 INFO:teuthology.orchestra.run.smithi017.stdout: altname enp7s0f0 2023-07-14T11:59:54.600 INFO:teuthology.orchestra.run.smithi017.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-14T11:59:54.600 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 0c:c4:7a:6b:fe:01 brd ff:ff:ff:ff:ff:ff 2023-07-14T11:59:54.600 INFO:teuthology.orchestra.run.smithi017.stdout: altname enp7s0f1 2023-07-14T11:59:54.600 INFO:teuthology.orchestra.run.smithi017.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-14T11:59:54.600 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 0c:c4:7a:59:d9:92 brd ff:ff:ff:ff:ff:ff 2023-07-14T11:59:54.600 INFO:teuthology.orchestra.run.smithi017.stdout: altname enp1s0f0 2023-07-14T11:59:54.600 INFO:teuthology.orchestra.run.smithi017.stdout: inet 172.21.15.17/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-07-14T11:59:54.601 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft 41548sec preferred_lft 41548sec 2023-07-14T11:59:54.601 INFO:teuthology.orchestra.run.smithi017.stdout: inet6 fe80::ec4:7aff:fe59:d992/64 scope link 2023-07-14T11:59:54.601 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft forever preferred_lft forever 2023-07-14T11:59:54.601 INFO:teuthology.orchestra.run.smithi017.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-14T11:59:54.601 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 0c:c4:7a:59:d9:93 brd ff:ff:ff:ff:ff:ff 2023-07-14T11:59:54.601 INFO:teuthology.orchestra.run.smithi017.stdout: altname enp1s0f1 2023-07-14T11:59:54.602 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-07-14T11:59:54.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:59:54.603 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec bash -c ' 2023-07-14T11:59:54.603 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2023-07-14T11:59:54.603 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link add name ceph-brx type bridge 2023-07-14T11:59:54.603 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip addr flush dev ceph-brx 2023-07-14T11:59:54.604 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link set ceph-brx up 2023-07-14T11:59:54.604 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-07-14T11:59:54.604 DEBUG:teuthology.orchestra.run.smithi017:> ') 2023-07-14T11:59:54.744 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:54 smithi017 ceph-mon[126339]: pgmap v282: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 2 op/s 2023-07-14T11:59:54.744 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:54 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/4222157508' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-14T11:59:54.861 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:54 smithi093 ceph-mon[128922]: pgmap v282: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 2 op/s 2023-07-14T11:59:54.861 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:54 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/4222157508' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-14T11:59:54.884 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:54 smithi097 ceph-mon[129313]: pgmap v282: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 2 op/s 2023-07-14T11:59:54.884 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:54 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/4222157508' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-14T11:59:55.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:59:55.094 DEBUG:teuthology.orchestra.run.smithi017:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-14T11:59:55.165 INFO:teuthology.orchestra.run.smithi017.stdout:1 2023-07-14T11:59:55.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:59:55.169 DEBUG:teuthology.orchestra.run.smithi017:> route 2023-07-14T11:59:55.231 INFO:teuthology.orchestra.run.smithi017.stdout:Kernel IP routing table 2023-07-14T11:59:55.232 INFO:teuthology.orchestra.run.smithi017.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-14T11:59:55.232 INFO:teuthology.orchestra.run.smithi017.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-07-14T11:59:55.232 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-07-14T11:59:55.232 INFO:teuthology.orchestra.run.smithi017.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-14T11:59:55.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:59:55.234 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec bash -c ' 2023-07-14T11:59:55.234 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2023-07-14T11:59:55.234 DEBUG:teuthology.orchestra.run.smithi017:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-07-14T11:59:55.235 DEBUG:teuthology.orchestra.run.smithi017:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-07-14T11:59:55.235 DEBUG:teuthology.orchestra.run.smithi017:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-07-14T11:59:55.235 DEBUG:teuthology.orchestra.run.smithi017:> ') 2023-07-14T11:59:55.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:59:55.910 DEBUG:teuthology.orchestra.run.smithi017:> ip netns list 2023-07-14T11:59:55.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:59:55.956 DEBUG:teuthology.orchestra.run.smithi017:> ip netns list-id 2023-07-14T11:59:56.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:59:56.039 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec bash -c ' 2023-07-14T11:59:56.039 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2023-07-14T11:59:56.040 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-07-14T11:59:56.040 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-07-14T11:59:56.040 DEBUG:teuthology.orchestra.run.smithi017:> ') 2023-07-14T11:59:56.225 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-07-14T11:59:56.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:59:56.226 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec bash -c ' 2023-07-14T11:59:56.226 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2023-07-14T11:59:56.227 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-07-14T11:59:56.227 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-07-14T11:59:56.228 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-07-14T11:59:56.228 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-07-14T11:59:56.228 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-07-14T11:59:56.228 DEBUG:teuthology.orchestra.run.smithi017:> ') 2023-07-14T11:59:56.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:59:56.533 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec bash -c ' 2023-07-14T11:59:56.533 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2023-07-14T11:59:56.533 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link set brx.0 up 2023-07-14T11:59:56.534 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link set dev brx.0 master ceph-brx 2023-07-14T11:59:56.534 DEBUG:teuthology.orchestra.run.smithi017:> ') 2023-07-14T11:59:56.712 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-07-14T11:59:56.712 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-14T11:59:56.713 DEBUG:teuthology.orchestra.run.smithi017:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-07-14T11:59:56.757 INFO:teuthology.orchestra.run.smithi017.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-07-14T11:59:56.757 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-14T11:59:56.758 DEBUG:teuthology.orchestra.run.smithi017:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-07-14T11:59:56.841 DEBUG:teuthology.orchestra.run.smithi017:> sudo modprobe fuse 2023-07-14T11:59:56.953 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-14T11:59:56.953 DEBUG:teuthology.orchestra.run.smithi017:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-14T11:59:56.997 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:56 smithi017 ceph-mon[126339]: pgmap v283: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s 2023-07-14T11:59:57.023 INFO:teuthology.orchestra.run.smithi017.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-14T11:59:57.025 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-14T11:59:57.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:59:57.026 DEBUG:teuthology.orchestra.run.smithi017:> ls /sys/fs/fuse/connections 2023-07-14T11:59:57.110 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-14T11:59:57.111 DEBUG:teuthology.orchestra.run.smithi017:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-07-14T11:59:57.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:56 smithi093 ceph-mon[128922]: pgmap v283: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s 2023-07-14T11:59:57.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:56 smithi097 ceph-mon[129313]: pgmap v283: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s 2023-07-14T11:59:57.154 DEBUG:teuthology.orchestra.run.smithi017:> sudo modprobe fuse 2023-07-14T11:59:57.230 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-14T11:59:57.231 DEBUG:teuthology.orchestra.run.smithi017:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-14T11:59:57.301 INFO:teuthology.orchestra.run.smithi017.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-14T11:59:57.304 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-14T11:59:57.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:59:57.305 DEBUG:teuthology.orchestra.run.smithi017:> ls /sys/fs/fuse/connections 2023-07-14T11:59:57.360 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi017.stderr:2023-07-14T11:59:57.378+0000 7ffa35eb75c0 -1 init, newargv = 0x55e6d7411c70 newargc=15 2023-07-14T11:59:57.360 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi017.stderr:ceph-fuse[163668]: starting ceph client 2023-07-14T11:59:57.369 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi017.stderr:ceph-fuse[163668]: starting fuse 2023-07-14T11:59:58.454 DEBUG:teuthology.orchestra.run.smithi017:> sudo modprobe fuse 2023-07-14T11:59:58.534 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-14T11:59:58.534 DEBUG:teuthology.orchestra.run.smithi017:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-14T11:59:58.605 INFO:teuthology.orchestra.run.smithi017.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-14T11:59:58.607 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-14T11:59:58.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:59:58.608 DEBUG:teuthology.orchestra.run.smithi017:> ls /sys/fs/fuse/connections 2023-07-14T11:59:58.652 INFO:teuthology.orchestra.run.smithi017.stdout:121 2023-07-14T11:59:58.653 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [121] 2023-07-14T11:59:58.654 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-14T11:59:58.654 DEBUG:teuthology.orchestra.run.smithi017:> import glob 2023-07-14T11:59:58.655 DEBUG:teuthology.orchestra.run.smithi017:> import re 2023-07-14T11:59:58.655 DEBUG:teuthology.orchestra.run.smithi017:> import os 2023-07-14T11:59:58.655 DEBUG:teuthology.orchestra.run.smithi017:> import subprocess 2023-07-14T11:59:58.655 DEBUG:teuthology.orchestra.run.smithi017:> 2023-07-14T11:59:58.656 DEBUG:teuthology.orchestra.run.smithi017:> def _find_admin_socket(client_name): 2023-07-14T11:59:58.656 DEBUG:teuthology.orchestra.run.smithi017:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-07-14T11:59:58.656 DEBUG:teuthology.orchestra.run.smithi017:> files = glob.glob(asok_path) 2023-07-14T11:59:58.657 DEBUG:teuthology.orchestra.run.smithi017:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-07-14T11:59:58.657 DEBUG:teuthology.orchestra.run.smithi017:> 2023-07-14T11:59:58.657 DEBUG:teuthology.orchestra.run.smithi017:> # Given a non-glob path, it better be there 2023-07-14T11:59:58.657 DEBUG:teuthology.orchestra.run.smithi017:> if "*" not in asok_path: 2023-07-14T11:59:58.658 DEBUG:teuthology.orchestra.run.smithi017:> assert(len(files) == 1) 2023-07-14T11:59:58.658 DEBUG:teuthology.orchestra.run.smithi017:> return files[0] 2023-07-14T11:59:58.658 DEBUG:teuthology.orchestra.run.smithi017:> 2023-07-14T11:59:58.658 DEBUG:teuthology.orchestra.run.smithi017:> for f in files: 2023-07-14T11:59:58.659 DEBUG:teuthology.orchestra.run.smithi017:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-14T11:59:58.659 DEBUG:teuthology.orchestra.run.smithi017:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-14T11:59:58.659 DEBUG:teuthology.orchestra.run.smithi017:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-14T11:59:58.660 DEBUG:teuthology.orchestra.run.smithi017:> contents = proc_f.read() 2023-07-14T11:59:58.660 DEBUG:teuthology.orchestra.run.smithi017:> if mountpoint in contents: 2023-07-14T11:59:58.660 DEBUG:teuthology.orchestra.run.smithi017:> return f 2023-07-14T11:59:58.660 DEBUG:teuthology.orchestra.run.smithi017:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-14T11:59:58.661 DEBUG:teuthology.orchestra.run.smithi017:> 2023-07-14T11:59:58.661 DEBUG:teuthology.orchestra.run.smithi017:> print(_find_admin_socket("client.admin")) 2023-07-14T11:59:58.661 DEBUG:teuthology.orchestra.run.smithi017:> ' 2023-07-14T11:59:58.841 INFO:teuthology.orchestra.run.smithi017.stdout:/var/run/ceph/ceph-client.admin.163668.asok 2023-07-14T11:59:58.894 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 11:59:58 smithi017 ceph-mon[126339]: pgmap v284: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 308 B/s wr, 0 op/s 2023-07-14T11:59:59.018 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.163668.asok 2023-07-14T11:59:59.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:59:59.019 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.163668.asok status 2023-07-14T11:59:59.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 11:59:58 smithi093 ceph-mon[128922]: pgmap v284: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 308 B/s wr, 0 op/s 2023-07-14T11:59:59.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 11:59:58 smithi097 ceph-mon[129313]: pgmap v284: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 308 B/s wr, 0 op/s 2023-07-14T11:59:59.252 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2023-07-14T11:59:59.252 INFO:teuthology.orchestra.run.smithi017.stdout: "metadata": { 2023-07-14T11:59:59.252 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", 2023-07-14T11:59:59.252 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-14T11:59:59.252 INFO:teuthology.orchestra.run.smithi017.stdout: "entity_id": "admin", 2023-07-14T11:59:59.253 INFO:teuthology.orchestra.run.smithi017.stdout: "hostname": "smithi017", 2023-07-14T11:59:59.253 INFO:teuthology.orchestra.run.smithi017.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-07-14T11:59:59.253 INFO:teuthology.orchestra.run.smithi017.stdout: "pid": "163668", 2023-07-14T11:59:59.253 INFO:teuthology.orchestra.run.smithi017.stdout: "root": "/" 2023-07-14T11:59:59.253 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-07-14T11:59:59.253 INFO:teuthology.orchestra.run.smithi017.stdout: "dentry_count": 0, 2023-07-14T11:59:59.253 INFO:teuthology.orchestra.run.smithi017.stdout: "dentry_pinned_count": 0, 2023-07-14T11:59:59.254 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 14679, 2023-07-14T11:59:59.254 INFO:teuthology.orchestra.run.smithi017.stdout: "inst": { 2023-07-14T11:59:59.254 INFO:teuthology.orchestra.run.smithi017.stdout: "name": { 2023-07-14T11:59:59.254 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "client", 2023-07-14T11:59:59.254 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 14679 2023-07-14T11:59:59.254 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-07-14T11:59:59.254 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": { 2023-07-14T11:59:59.255 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2023-07-14T11:59:59.255 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "192.168.0.1:0", 2023-07-14T11:59:59.255 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 829062619 2023-07-14T11:59:59.255 INFO:teuthology.orchestra.run.smithi017.stdout: } 2023-07-14T11:59:59.255 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-07-14T11:59:59.255 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": { 2023-07-14T11:59:59.255 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2023-07-14T11:59:59.256 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "192.168.0.1:0", 2023-07-14T11:59:59.256 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 829062619 2023-07-14T11:59:59.256 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-07-14T11:59:59.256 INFO:teuthology.orchestra.run.smithi017.stdout: "inst_str": "client.14679 192.168.0.1:0/829062619", 2023-07-14T11:59:59.256 INFO:teuthology.orchestra.run.smithi017.stdout: "addr_str": "192.168.0.1:0/829062619", 2023-07-14T11:59:59.256 INFO:teuthology.orchestra.run.smithi017.stdout: "inode_count": 1, 2023-07-14T11:59:59.256 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_epoch": 17, 2023-07-14T11:59:59.257 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_epoch": 88, 2023-07-14T11:59:59.257 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_epoch_barrier": 0, 2023-07-14T11:59:59.257 INFO:teuthology.orchestra.run.smithi017.stdout: "blocklisted": false, 2023-07-14T11:59:59.257 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs" 2023-07-14T11:59:59.257 INFO:teuthology.orchestra.run.smithi017.stdout:} 2023-07-14T11:59:59.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:59:59.266 DEBUG:teuthology.orchestra.run.smithi017:> stat --file-system '--printf=%T 2023-07-14T11:59:59.267 DEBUG:teuthology.orchestra.run.smithi017:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-07-14T11:59:59.311 INFO:teuthology.orchestra.run.smithi017.stdout:fuseblk 2023-07-14T11:59:59.311 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-07-14T11:59:59.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:59:59.312 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-07-14T11:59:59.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T11:59:59.423 DEBUG:teuthology.orchestra.run.smithi017:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-07-14T11:59:59.542 INFO:teuthology.orchestra.run.smithi017.stdout:# file: . 2023-07-14T11:59:59.542 INFO:teuthology.orchestra.run.smithi017.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-07-14T11:59:59.543 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T11:59:59.544 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2023-07-14T11:59:59.545 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T11:59:59.589 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T11:59:59.833 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:59.852+0000 7fa26c3ff700 1 -- 172.21.15.17:0/1302544145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2640ffba0 msgr2=0x7fa2640fffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:59.833 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:59.852+0000 7fa26c3ff700 1 --2- 172.21.15.17:0/1302544145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2640ffba0 0x7fa2640fffb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa260009990 tx=0x7fa260009ca0 comp rx=0 tx=0).stop 2023-07-14T11:59:59.834 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:59.852+0000 7fa26c3ff700 1 -- 172.21.15.17:0/1302544145 shutdown_connections 2023-07-14T11:59:59.834 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:59.852+0000 7fa26c3ff700 1 --2- 172.21.15.17:0/1302544145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2641017e0 0x7fa264108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:59.834 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:59.852+0000 7fa26c3ff700 1 --2- 172.21.15.17:0/1302544145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2640ffba0 0x7fa2640fffb0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:59.834 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:59.852+0000 7fa26c3ff700 1 --2- 172.21.15.17:0/1302544145 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa264100e30 0x7fa2641012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:59.834 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:59.852+0000 7fa26c3ff700 1 -- 172.21.15.17:0/1302544145 >> 172.21.15.17:0/1302544145 conn(0x7fa2640fb1c0 msgr2=0x7fa2640fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T11:59:59.834 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:59.852+0000 7fa26c3ff700 1 -- 172.21.15.17:0/1302544145 shutdown_connections 2023-07-14T11:59:59.834 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:59.852+0000 7fa26c3ff700 1 -- 172.21.15.17:0/1302544145 wait complete. 2023-07-14T11:59:59.837 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:59.853+0000 7fa26c3ff700 1 Processor -- start 2023-07-14T11:59:59.837 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:59.853+0000 7fa26c3ff700 1 -- start start 2023-07-14T11:59:59.837 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:59.854+0000 7fa26c3ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2640ffba0 0x7fa2640742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:59.837 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:59.854+0000 7fa26c3ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa264100e30 0x7fa264072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:59.837 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:59.854+0000 7fa26c3ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2641017e0 0x7fa264072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:59.838 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:59.854+0000 7fa26c3ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2640735e0 con 0x7fa2641017e0 2023-07-14T11:59:59.838 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:59.854+0000 7fa26c3ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa264073750 con 0x7fa264100e30 2023-07-14T11:59:59.838 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:59.854+0000 7fa26c3ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa26419f990 con 0x7fa2640ffba0 2023-07-14T11:59:59.838 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:59.854+0000 7fa26a99c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2641017e0 0x7fa264072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:59.838 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:59.854+0000 7fa26a99c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2641017e0 0x7fa264072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34628/0 (socket says 172.21.15.17:34628) 2023-07-14T11:59:59.838 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:59.854+0000 7fa26a99c700 1 -- 172.21.15.17:0/47028281 learned_addr learned my addr 172.21.15.17:0/47028281 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T11:59:59.838 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:59.854+0000 7fa26a19b700 1 --2- 172.21.15.17:0/47028281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2640ffba0 0x7fa2640742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:59.839 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:59.854+0000 7fa26999a700 1 --2- 172.21.15.17:0/47028281 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa264100e30 0x7fa264072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:59.839 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:59.854+0000 7fa26a99c700 1 -- 172.21.15.17:0/47028281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2640ffba0 msgr2=0x7fa2640742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:59.839 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:59.854+0000 7fa26a99c700 1 --2- 172.21.15.17:0/47028281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2640ffba0 0x7fa2640742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:59.840 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:59.855+0000 7fa26a99c700 1 -- 172.21.15.17:0/47028281 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa264100e30 msgr2=0x7fa264072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T11:59:59.840 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:59.855+0000 7fa26a99c700 1 --2- 172.21.15.17:0/47028281 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa264100e30 0x7fa264072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T11:59:59.840 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:59.855+0000 7fa26a99c700 1 -- 172.21.15.17:0/47028281 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2600095f0 con 0x7fa2641017e0 2023-07-14T11:59:59.841 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:59.855+0000 7fa26a99c700 1 --2- 172.21.15.17:0/47028281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2641017e0 0x7fa264072e40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa25c00ef30 tx=0x7fa25c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:59.841 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:59.855+0000 7fa25b7fe700 1 -- 172.21.15.17:0/47028281 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25c018070 con 0x7fa2641017e0 2023-07-14T11:59:59.841 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:59.855+0000 7fa25b7fe700 1 -- 172.21.15.17:0/47028281 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa25c00f040 con 0x7fa2641017e0 2023-07-14T11:59:59.841 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:59.855+0000 7fa25b7fe700 1 -- 172.21.15.17:0/47028281 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25c00cd00 con 0x7fa2641017e0 2023-07-14T11:59:59.842 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:59.855+0000 7fa26c3ff700 1 -- 172.21.15.17:0/47028281 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa26419fc10 con 0x7fa2641017e0 2023-07-14T11:59:59.842 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:59.855+0000 7fa26c3ff700 1 -- 172.21.15.17:0/47028281 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2641a00d0 con 0x7fa2641017e0 2023-07-14T11:59:59.842 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:59.856+0000 7fa26c3ff700 1 -- 172.21.15.17:0/47028281 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa264106bc0 con 0x7fa2641017e0 2023-07-14T11:59:59.844 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:59.858+0000 7fa25b7fe700 1 -- 172.21.15.17:0/47028281 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa25c003a00 con 0x7fa2641017e0 2023-07-14T11:59:59.844 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:59.860+0000 7fa25b7fe700 1 --2- 172.21.15.17:0/47028281 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fa2500a1150 0x7fa2500a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T11:59:59.844 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:59.860+0000 7fa25b7fe700 1 -- 172.21.15.17:0/47028281 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa25c0cc610 con 0x7fa2641017e0 2023-07-14T11:59:59.844 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:59.860+0000 7fa26a19b700 1 --2- 172.21.15.17:0/47028281 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fa2500a1150 0x7fa2500a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T11:59:59.844 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:59.861+0000 7fa26a19b700 1 --2- 172.21.15.17:0/47028281 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fa2500a1150 0x7fa2500a3600 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa26000b2e0 tx=0x7fa260005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T11:59:59.845 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T11:59:59.864+0000 7fa25b7fe700 1 -- 172.21.15.17:0/47028281 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa25c095ba0 con 0x7fa2641017e0 2023-07-14T12:00:00.054 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.074+0000 7fa26c3ff700 1 -- 172.21.15.17:0/47028281 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa26402c850 con 0x7fa2641017e0 2023-07-14T12:00:00.056 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.076+0000 7fa25b7fe700 1 -- 172.21.15.17:0/47028281 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23866 (secure 0 0 0) 0x7fa25c0991c0 con 0x7fa2641017e0 2023-07-14T12:00:00.057 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T12:00:00.057 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":88,"fsid":"b3c386cc-223c-11ee-9b33-001a4aab830c","created":"2023-07-14T11:51:59.992960+0000","modified":"2023-07-14T11:59:48.295704+0000","last_up_change":"2023-07-14T11:58:55.037769+0000","last_in_change":"2023-07-14T11:58:37.114870+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:55:28.237002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:59:41.560502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T11:59:42.725461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:59:46.267029+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"57003438-a83c-4c8e-a029-56414740895c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6803","nonce":3588366384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6805","nonce":3588366384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6809","nonce":3588366384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6807","nonce":3588366384}]},"public_addr":"172.21.15.17:6803/3588366384","cluster_addr":"172.21.15.17:6805/3588366384","heartbeat_back_addr":"172.21.15.17:6809/3588366384","heartbeat_front_addr":"172.21.15.17:6807/3588366384","state":["exists","up"]},{"osd":1,"uuid":"f56b6337-d8a9-4411-b9d8-b6863dc98556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6801","nonce":3147203064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6803","nonce":3147203064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6807","nonce":3147203064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6805","nonce":3147203064}]},"public_addr":"172.21.15.93:6801/3147203064","cluster_addr":"172.21.15.93:6803/3147203064","heartbeat_back_addr":"172.21.15.93:6807/3147203064","heartbeat_front_addr":"172.21.15.93:6805/3147203064","state":["exists","up"]},{"osd":2,"uuid":"fc34bb7f-551f-46e1-99c0-59924d6c1346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6801","nonce":1877269815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6803","nonce":1877269815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6807","nonce":1877269815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6805","nonce":1877269815}]},"public_addr":"172.21.15.97:6801/1877269815","cluster_addr":"172.21.15.97:6803/1877269815","heartbeat_back_addr":"172.21.15.97:6807/1877269815","heartbeat_front_addr":"172.21.15.97:6805/1877269815","state":["exists","up"]},{"osd":3,"uuid":"37a1583c-4fb8-412a-a0c8-f83b1ff26a91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6811","nonce":2784619821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6813","nonce":2784619821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6817","nonce":2784619821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6815","nonce":2784619821}]},"public_addr":"172.21.15.17:6811/2784619821","cluster_addr":"172.21.15.17:6813/2784619821","heartbeat_back_addr":"172.21.15.17:6817/2784619821","heartbeat_front_addr":"172.21.15.17:6815/2784619821","state":["exists","up"]},{"osd":4,"uuid":"f4b9f377-d519-4b14-ac4a-f1debaeed9f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6809","nonce":3151801173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6811","nonce":3151801173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6815","nonce":3151801173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6813","nonce":3151801173}]},"public_addr":"172.21.15.93:6809/3151801173","cluster_addr":"172.21.15.93:6811/3151801173","heartbeat_back_addr":"172.21.15.93:6815/3151801173","heartbeat_front_addr":"172.21.15.93:6813/3151801173","state":["exists","up"]},{"osd":5,"uuid":"2d2af354-0535-4ef9-b166-3ef219498db8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6809","nonce":2184413924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6811","nonce":2184413924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6815","nonce":2184413924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6813","nonce":2184413924}]},"public_addr":"172.21.15.97:6809/2184413924","cluster_addr":"172.21.15.97:6811/2184413924","heartbeat_back_addr":"172.21.15.97:6815/2184413924","heartbeat_front_addr":"172.21.15.97:6813/2184413924","state":["exists","up"]},{"osd":6,"uuid":"9cdf0c4b-62dc-4168-864e-a9d1650a9ea8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6819","nonce":50486581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6821","nonce":50486581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6825","nonce":50486581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6823","nonce":50486581}]},"public_addr":"172.21.15.17:6819/50486581","cluster_addr":"172.21.15.17:6821/50486581","heartbeat_back_addr":"172.21.15.17:6825/50486581","heartbeat_front_addr":"172.21.15.17:6823/50486581","state":["exists","up"]},{"osd":7,"uuid":"09f60823-5491-45cd-9324-6ae0d56ce262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6817","nonce":2596912392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6819","nonce":2596912392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6823","nonce":2596912392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6821","nonce":2596912392}]},"public_addr":"172.21.15.93:6817/2596912392","cluster_addr":"172.21.15.93:6819/2596912392","heartbeat_back_addr":"172.21.15.93:6823/2596912392","heartbeat_front_addr":"172.21.15.93:6821/2596912392","state":["exists","up"]},{"osd":8,"uuid":"8c5771c9-718e-4313-a2b6-5e1c37c53f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6817","nonce":1965492859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6819","nonce":1965492859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6823","nonce":1965492859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6821","nonce":1965492859}]},"public_addr":"172.21.15.97:6817/1965492859","cluster_addr":"172.21.15.97:6819/1965492859","heartbeat_back_addr":"172.21.15.97:6823/1965492859","heartbeat_front_addr":"172.21.15.97:6821/1965492859","state":["exists","up"]},{"osd":9,"uuid":"1197b8a3-c09f-4c57-aba1-9e988a1620b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6827","nonce":2512645700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6829","nonce":2512645700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6833","nonce":2512645700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6831","nonce":2512645700}]},"public_addr":"172.21.15.17:6827/2512645700","cluster_addr":"172.21.15.17:6829/2512645700","heartbeat_back_addr":"172.21.15.17:6833/2512645700","heartbeat_front_addr":"172.21.15.17:6831/2512645700","state":["exists","up"]},{"osd":10,"uuid":"f99918c2-cdd8-4a7a-be2e-d1d6e0f828a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6825","nonce":264429558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6827","nonce":264429558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6831","nonce":264429558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6829","nonce":264429558}]},"public_addr":"172.21.15.93:6825/264429558","cluster_addr":"172.21.15.93:6827/264429558","heartbeat_back_addr":"172.21.15.93:6831/264429558","heartbeat_front_addr":"172.21.15.93:6829/264429558","state":["exists","up"]},{"osd":11,"uuid":"6bbb5b79-8d4d-441f-9583-203df6c78cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6825","nonce":3389913754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6827","nonce":3389913754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6831","nonce":3389913754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6829","nonce":3389913754}]},"public_addr":"172.21.15.97:6825/3389913754","cluster_addr":"172.21.15.97:6827/3389913754","heartbeat_back_addr":"172.21.15.97:6831/3389913754","heartbeat_front_addr":"172.21.15.97:6829/3389913754","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:54:41.079618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:03.207899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:24.136961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:50.392609+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:56:12.136237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:56:33.212314+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:00.243646+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:22.059234+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:43.232399+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:10.043896+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:31.910780+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:53.158983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/2084033276":"2023-07-15T11:53:05.639597+0000","172.21.15.17:6801/1059284716":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/3651785636":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/1373538024":"2023-07-15T11:52:37.887946+0000","172.21.15.17:0/3560395020":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6800/1059284716":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/4262912861":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/2258375202":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6800/2804227742":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6801/2804227742":"2023-07-15T11:52:37.887946+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T12:00:00.058 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.078+0000 7fa26c3ff700 1 -- 172.21.15.17:0/47028281 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fa2500a1150 msgr2=0x7fa2500a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:00.059 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.078+0000 7fa26c3ff700 1 --2- 172.21.15.17:0/47028281 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fa2500a1150 0x7fa2500a3600 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa26000b2e0 tx=0x7fa260005fb0 comp rx=0 tx=0).stop 2023-07-14T12:00:00.059 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.078+0000 7fa26c3ff700 1 -- 172.21.15.17:0/47028281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2641017e0 msgr2=0x7fa264072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:00.059 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.078+0000 7fa26c3ff700 1 --2- 172.21.15.17:0/47028281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2641017e0 0x7fa264072e40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa25c00ef30 tx=0x7fa25c00c3a0 comp rx=0 tx=0).stop 2023-07-14T12:00:00.059 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.078+0000 7fa26c3ff700 1 -- 172.21.15.17:0/47028281 shutdown_connections 2023-07-14T12:00:00.059 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.078+0000 7fa26c3ff700 1 --2- 172.21.15.17:0/47028281 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fa2500a1150 0x7fa2500a3600 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:00.059 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.078+0000 7fa26c3ff700 1 --2- 172.21.15.17:0/47028281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2641017e0 0x7fa264072e40 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:00.059 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.078+0000 7fa26c3ff700 1 --2- 172.21.15.17:0/47028281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2640ffba0 0x7fa2640742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:00.059 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.078+0000 7fa26c3ff700 1 --2- 172.21.15.17:0/47028281 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa264100e30 0x7fa264072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:00.060 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.078+0000 7fa26c3ff700 1 -- 172.21.15.17:0/47028281 >> 172.21.15.17:0/47028281 conn(0x7fa2640fb1c0 msgr2=0x7fa2640fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:00.060 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.078+0000 7fa26c3ff700 1 -- 172.21.15.17:0/47028281 shutdown_connections 2023-07-14T12:00:00.060 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.078+0000 7fa26c3ff700 1 -- 172.21.15.17:0/47028281 wait complete. 2023-07-14T12:00:00.078 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:55:28.237002+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T11:59:41.560502+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T11:59:42.725461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-14T11:59:46.267029+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-14T12:00:00.079 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T12:00:00.291 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.310+0000 7f753f5ff700 1 -- 172.21.15.17:0/2431265416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7538072f70 msgr2=0x7f75380733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:00.291 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.310+0000 7f753f5ff700 1 --2- 172.21.15.17:0/2431265416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7538072f70 0x7f75380733e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f75280098f0 tx=0x7f7528009c00 comp rx=0 tx=0).stop 2023-07-14T12:00:00.291 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.310+0000 7f753f5ff700 1 -- 172.21.15.17:0/2431265416 shutdown_connections 2023-07-14T12:00:00.291 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.310+0000 7f753f5ff700 1 --2- 172.21.15.17:0/2431265416 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7538107aa0 0x7f7538109f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:00.292 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.310+0000 7f753f5ff700 1 --2- 172.21.15.17:0/2431265416 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7538074550 0x7f75380729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:00.292 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.310+0000 7f753f5ff700 1 --2- 172.21.15.17:0/2431265416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7538072f70 0x7f75380733e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:00.292 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.310+0000 7f753f5ff700 1 -- 172.21.15.17:0/2431265416 >> 172.21.15.17:0/2431265416 conn(0x7f75380fb200 msgr2=0x7f75380fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:00.292 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.310+0000 7f753f5ff700 1 -- 172.21.15.17:0/2431265416 shutdown_connections 2023-07-14T12:00:00.292 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.310+0000 7f753f5ff700 1 -- 172.21.15.17:0/2431265416 wait complete. 2023-07-14T12:00:00.294 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.312+0000 7f753f5ff700 1 Processor -- start 2023-07-14T12:00:00.295 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.312+0000 7f753f5ff700 1 -- start start 2023-07-14T12:00:00.295 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.312+0000 7f753f5ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7538072f70 0x7f7538197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:00.295 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.312+0000 7f753f5ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7538074550 0x7f7538198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:00.295 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.312+0000 7f753f5ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7538107aa0 0x7f753819f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:00.295 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.312+0000 7f753f5ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753819f6d0 con 0x7f7538107aa0 2023-07-14T12:00:00.296 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.312+0000 7f753f5ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753819f810 con 0x7f7538072f70 2023-07-14T12:00:00.296 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.312+0000 7f753f5ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753819f9a0 con 0x7f7538074550 2023-07-14T12:00:00.296 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.313+0000 7f753db9c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7538107aa0 0x7f753819f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:00.296 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.313+0000 7f753cb9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7538074550 0x7f7538198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:00.296 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.313+0000 7f753db9c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7538107aa0 0x7f753819f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34652/0 (socket says 172.21.15.17:34652) 2023-07-14T12:00:00.296 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.313+0000 7f753d39b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7538072f70 0x7f7538197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:00.296 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.313+0000 7f753db9c700 1 -- 172.21.15.17:0/901614537 learned_addr learned my addr 172.21.15.17:0/901614537 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:00:00.296 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.313+0000 7f753cb9a700 1 -- 172.21.15.17:0/901614537 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7538072f70 msgr2=0x7f7538197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:00.297 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.313+0000 7f753cb9a700 1 --2- 172.21.15.17:0/901614537 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7538072f70 0x7f7538197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:00.297 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.313+0000 7f753cb9a700 1 -- 172.21.15.17:0/901614537 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7538107aa0 msgr2=0x7f753819f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:00.297 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.313+0000 7f753cb9a700 1 --2- 172.21.15.17:0/901614537 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7538107aa0 0x7f753819f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:00.297 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.313+0000 7f753cb9a700 1 -- 172.21.15.17:0/901614537 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75340095f0 con 0x7f7538074550 2023-07-14T12:00:00.297 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.313+0000 7f753db9c700 1 --2- 172.21.15.17:0/901614537 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7538107aa0 0x7f753819f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T12:00:00.297 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.313+0000 7f753cb9a700 1 --2- 172.21.15.17:0/901614537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7538074550 0x7f7538198090 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7528000c00 tx=0x7f752800b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:00.297 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.313+0000 7f752e7fc700 1 -- 172.21.15.17:0/901614537 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752801c070 con 0x7f7538074550 2023-07-14T12:00:00.298 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.314+0000 7f752e7fc700 1 -- 172.21.15.17:0/901614537 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f752800f7d0 con 0x7f7538074550 2023-07-14T12:00:00.298 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.314+0000 7f752e7fc700 1 -- 172.21.15.17:0/901614537 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7528003990 con 0x7f7538074550 2023-07-14T12:00:00.298 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.314+0000 7f753f5ff700 1 -- 172.21.15.17:0/901614537 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7528009520 con 0x7f7538074550 2023-07-14T12:00:00.298 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.314+0000 7f753f5ff700 1 -- 172.21.15.17:0/901614537 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f753819fe60 con 0x7f7538074550 2023-07-14T12:00:00.299 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.315+0000 7f753f5ff700 1 -- 172.21.15.17:0/901614537 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7538106150 con 0x7f7538074550 2023-07-14T12:00:00.299 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.317+0000 7f752e7fc700 1 -- 172.21.15.17:0/901614537 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7528020070 con 0x7f7538074550 2023-07-14T12:00:00.299 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.318+0000 7f752e7fc700 1 --2- 172.21.15.17:0/901614537 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f75240a1150 0x7f75240a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:00.299 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.318+0000 7f752e7fc700 1 -- 172.21.15.17:0/901614537 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f75280cd7f0 con 0x7f7538074550 2023-07-14T12:00:00.299 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.318+0000 7f753d39b700 1 --2- 172.21.15.17:0/901614537 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f75240a1150 0x7f75240a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:00.300 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.319+0000 7f753d39b700 1 --2- 172.21.15.17:0/901614537 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f75240a1150 0x7f75240a3600 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7538074920 tx=0x7f7534009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:00.301 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.320+0000 7f752e7fc700 1 -- 172.21.15.17:0/901614537 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7528096d80 con 0x7f7538074550 2023-07-14T12:00:00.512 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.531+0000 7f753f5ff700 1 -- 172.21.15.17:0/901614537 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f753802c9e0 con 0x7f7538074550 2023-07-14T12:00:00.513 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.532+0000 7f752e7fc700 1 -- 172.21.15.17:0/901614537 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f752809a3a0 con 0x7f7538074550 2023-07-14T12:00:00.513 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2023-07-14T12:00:00.515 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.534+0000 7f753f5ff700 1 -- 172.21.15.17:0/901614537 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f75240a1150 msgr2=0x7f75240a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:00.515 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.534+0000 7f753f5ff700 1 --2- 172.21.15.17:0/901614537 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f75240a1150 0x7f75240a3600 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7538074920 tx=0x7f7534009310 comp rx=0 tx=0).stop 2023-07-14T12:00:00.515 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.534+0000 7f753f5ff700 1 -- 172.21.15.17:0/901614537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7538074550 msgr2=0x7f7538198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:00.516 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.534+0000 7f753f5ff700 1 --2- 172.21.15.17:0/901614537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7538074550 0x7f7538198090 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7528000c00 tx=0x7f752800b790 comp rx=0 tx=0).stop 2023-07-14T12:00:00.516 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.534+0000 7f753f5ff700 1 -- 172.21.15.17:0/901614537 shutdown_connections 2023-07-14T12:00:00.516 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.534+0000 7f753f5ff700 1 --2- 172.21.15.17:0/901614537 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f75240a1150 0x7f75240a3600 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:00.516 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.534+0000 7f753f5ff700 1 --2- 172.21.15.17:0/901614537 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7538072f70 0x7f7538197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:00.516 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.534+0000 7f753f5ff700 1 --2- 172.21.15.17:0/901614537 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7538107aa0 0x7f753819f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:00.516 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.534+0000 7f753f5ff700 1 --2- 172.21.15.17:0/901614537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7538074550 0x7f7538198090 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:00.516 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.534+0000 7f753f5ff700 1 -- 172.21.15.17:0/901614537 >> 172.21.15.17:0/901614537 conn(0x7f75380fb200 msgr2=0x7f75380fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:00.517 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.535+0000 7f753f5ff700 1 -- 172.21.15.17:0/901614537 shutdown_connections 2023-07-14T12:00:00.517 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.535+0000 7f753f5ff700 1 -- 172.21.15.17:0/901614537 wait complete. 2023-07-14T12:00:00.533 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T12:00:00.742 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.761+0000 7f4800fff700 1 -- 172.21.15.17:0/3272467876 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47fc101800 msgr2=0x7f47fc1080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:00.742 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.761+0000 7f4800fff700 1 --2- 172.21.15.17:0/3272467876 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47fc101800 0x7f47fc1080a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f47f400b0a0 tx=0x7f47f400b3b0 comp rx=0 tx=0).stop 2023-07-14T12:00:00.742 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.761+0000 7f4800fff700 1 -- 172.21.15.17:0/3272467876 shutdown_connections 2023-07-14T12:00:00.743 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.761+0000 7f4800fff700 1 --2- 172.21.15.17:0/3272467876 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47fc101800 0x7f47fc1080a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:00.743 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.761+0000 7f4800fff700 1 --2- 172.21.15.17:0/3272467876 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47fc0ffbc0 0x7f47fc0fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:00.743 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.761+0000 7f4800fff700 1 --2- 172.21.15.17:0/3272467876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47fc100e50 0x7f47fc1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:00.743 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.761+0000 7f4800fff700 1 -- 172.21.15.17:0/3272467876 >> 172.21.15.17:0/3272467876 conn(0x7f47fc0fb200 msgr2=0x7f47fc0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:00.744 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.761+0000 7f4800fff700 1 -- 172.21.15.17:0/3272467876 shutdown_connections 2023-07-14T12:00:00.744 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.761+0000 7f4800fff700 1 -- 172.21.15.17:0/3272467876 wait complete. 2023-07-14T12:00:00.748 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.762+0000 7f4800fff700 1 Processor -- start 2023-07-14T12:00:00.748 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.762+0000 7f4800fff700 1 -- start start 2023-07-14T12:00:00.749 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.763+0000 7f4800fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47fc0ffbc0 0x7f47fc197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:00.750 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.763+0000 7f4800fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47fc100e50 0x7f47fc198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:00.750 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.763+0000 7f4800fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47fc101800 0x7f47fc19f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:00.750 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.763+0000 7f4800fff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47fc19f770 con 0x7f47fc100e50 2023-07-14T12:00:00.750 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.763+0000 7f4800fff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47fc19f8b0 con 0x7f47fc0ffbc0 2023-07-14T12:00:00.750 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.763+0000 7f4800fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47fc19fa40 con 0x7f47fc101800 2023-07-14T12:00:00.750 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.763+0000 7f47f9d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47fc100e50 0x7f47fc198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:00.751 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.763+0000 7f47f9d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47fc100e50 0x7f47fc198130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34680/0 (socket says 172.21.15.17:34680) 2023-07-14T12:00:00.751 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.763+0000 7f47f9d9b700 1 -- 172.21.15.17:0/3759411032 learned_addr learned my addr 172.21.15.17:0/3759411032 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:00:00.751 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.763+0000 7f47fa59c700 1 --2- 172.21.15.17:0/3759411032 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47fc0ffbc0 0x7f47fc197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:00.751 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.763+0000 7f47fad9d700 1 --2- 172.21.15.17:0/3759411032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47fc101800 0x7f47fc19f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:00.751 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.763+0000 7f47f9d9b700 1 -- 172.21.15.17:0/3759411032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47fc101800 msgr2=0x7f47fc19f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:00.751 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.763+0000 7f47f9d9b700 1 --2- 172.21.15.17:0/3759411032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47fc101800 0x7f47fc19f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:00.751 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.763+0000 7f47f9d9b700 1 -- 172.21.15.17:0/3759411032 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47fc0ffbc0 msgr2=0x7f47fc197bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:00.752 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.763+0000 7f47f9d9b700 1 --2- 172.21.15.17:0/3759411032 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47fc0ffbc0 0x7f47fc197bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:00.752 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.763+0000 7f47f9d9b700 1 -- 172.21.15.17:0/3759411032 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f4009d00 con 0x7f47fc100e50 2023-07-14T12:00:00.752 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.764+0000 7f47f9d9b700 1 --2- 172.21.15.17:0/3759411032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47fc100e50 0x7f47fc198130 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f47f000e9a0 tx=0x7f47f000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:00.752 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.764+0000 7f47eb7fe700 1 -- 172.21.15.17:0/3759411032 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f0009790 con 0x7f47fc100e50 2023-07-14T12:00:00.752 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.764+0000 7f4800fff700 1 -- 172.21.15.17:0/3759411032 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47fc19fd20 con 0x7f47fc100e50 2023-07-14T12:00:00.753 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.764+0000 7f47eb7fe700 1 -- 172.21.15.17:0/3759411032 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47f0010430 con 0x7f47fc100e50 2023-07-14T12:00:00.753 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.764+0000 7f47eb7fe700 1 -- 172.21.15.17:0/3759411032 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f0018680 con 0x7f47fc100e50 2023-07-14T12:00:00.753 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.764+0000 7f4800fff700 1 -- 172.21.15.17:0/3759411032 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47fc1a0220 con 0x7f47fc100e50 2023-07-14T12:00:00.753 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.765+0000 7f4800fff700 1 -- 172.21.15.17:0/3759411032 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47fc10b3e0 con 0x7f47fc100e50 2023-07-14T12:00:00.753 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.767+0000 7f47eb7fe700 1 -- 172.21.15.17:0/3759411032 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f47f00105a0 con 0x7f47fc100e50 2023-07-14T12:00:00.754 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.768+0000 7f47eb7fe700 1 --2- 172.21.15.17:0/3759411032 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f47e40a11d0 0x7f47e40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:00.754 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.768+0000 7f47eb7fe700 1 -- 172.21.15.17:0/3759411032 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f47f0014070 con 0x7f47fc100e50 2023-07-14T12:00:00.754 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.769+0000 7f47fa59c700 1 --2- 172.21.15.17:0/3759411032 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f47e40a11d0 0x7f47e40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:00.754 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.769+0000 7f47fa59c700 1 --2- 172.21.15.17:0/3759411032 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f47e40a11d0 0x7f47e40a3680 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f47ec009c60 tx=0x7f47ec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:00.755 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.772+0000 7f47eb7fe700 1 -- 172.21.15.17:0/3759411032 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f47f0051790 con 0x7f47fc100e50 2023-07-14T12:00:00.894 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:00 smithi017 ceph-mon[126339]: pgmap v285: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 363 B/s wr, 0 op/s 2023-07-14T12:00:00.894 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:00 smithi017 ceph-mon[126339]: overall HEALTH_OK 2023-07-14T12:00:00.894 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:00 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/47028281' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T12:00:00.894 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:00 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/901614537' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T12:00:00.964 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.983+0000 7f4800fff700 1 -- 172.21.15.17:0/3759411032 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f47fc1a0830 con 0x7f47fc100e50 2023-07-14T12:00:00.965 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.984+0000 7f47eb7fe700 1 -- 172.21.15.17:0/3759411032 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f47f00961b0 con 0x7f47fc100e50 2023-07-14T12:00:00.965 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2023-07-14T12:00:00.967 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.986+0000 7f4800fff700 1 -- 172.21.15.17:0/3759411032 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f47e40a11d0 msgr2=0x7f47e40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:00.967 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.986+0000 7f4800fff700 1 --2- 172.21.15.17:0/3759411032 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f47e40a11d0 0x7f47e40a3680 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f47ec009c60 tx=0x7f47ec009260 comp rx=0 tx=0).stop 2023-07-14T12:00:00.967 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.986+0000 7f4800fff700 1 -- 172.21.15.17:0/3759411032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47fc100e50 msgr2=0x7f47fc198130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:00.968 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.986+0000 7f4800fff700 1 --2- 172.21.15.17:0/3759411032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47fc100e50 0x7f47fc198130 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f47f000e9a0 tx=0x7f47f000ecb0 comp rx=0 tx=0).stop 2023-07-14T12:00:00.968 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.986+0000 7f4800fff700 1 -- 172.21.15.17:0/3759411032 shutdown_connections 2023-07-14T12:00:00.968 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.986+0000 7f4800fff700 1 --2- 172.21.15.17:0/3759411032 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f47e40a11d0 0x7f47e40a3680 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:00.968 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.986+0000 7f4800fff700 1 --2- 172.21.15.17:0/3759411032 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47fc0ffbc0 0x7f47fc197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:00.968 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.986+0000 7f4800fff700 1 --2- 172.21.15.17:0/3759411032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47fc101800 0x7f47fc19f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:00.968 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.987+0000 7f4800fff700 1 --2- 172.21.15.17:0/3759411032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47fc100e50 0x7f47fc198130 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:00.968 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.987+0000 7f4800fff700 1 -- 172.21.15.17:0/3759411032 >> 172.21.15.17:0/3759411032 conn(0x7f47fc0fb200 msgr2=0x7f47fc0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:00.969 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.987+0000 7f4800fff700 1 -- 172.21.15.17:0/3759411032 shutdown_connections 2023-07-14T12:00:00.969 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:00.987+0000 7f4800fff700 1 -- 172.21.15.17:0/3759411032 wait complete. 2023-07-14T12:00:00.985 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T12:00:01.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:00 smithi093 ceph-mon[128922]: pgmap v285: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 363 B/s wr, 0 op/s 2023-07-14T12:00:01.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:00 smithi093 ceph-mon[128922]: overall HEALTH_OK 2023-07-14T12:00:01.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:00 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/47028281' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T12:00:01.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:00 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/901614537' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T12:00:01.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:00 smithi097 ceph-mon[129313]: pgmap v285: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 363 B/s wr, 0 op/s 2023-07-14T12:00:01.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:00 smithi097 ceph-mon[129313]: overall HEALTH_OK 2023-07-14T12:00:01.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:00 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/47028281' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T12:00:01.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:00 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/901614537' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T12:00:01.192 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.211+0000 7f7ac47ff700 1 -- 172.21.15.17:0/1815092448 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7abc0ffbe0 msgr2=0x7f7abc0ffff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:01.192 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.211+0000 7f7ac47ff700 1 --2- 172.21.15.17:0/1815092448 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7abc0ffbe0 0x7f7abc0ffff0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8009990 tx=0x7f7ab8009ca0 comp rx=0 tx=0).stop 2023-07-14T12:00:01.193 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.211+0000 7f7ac47ff700 1 -- 172.21.15.17:0/1815092448 shutdown_connections 2023-07-14T12:00:01.193 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.211+0000 7f7ac47ff700 1 --2- 172.21.15.17:0/1815092448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7abc101760 0x7f7abc108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:01.193 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.211+0000 7f7ac47ff700 1 --2- 172.21.15.17:0/1815092448 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7abc0ffbe0 0x7f7abc0ffff0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:01.193 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.211+0000 7f7ac47ff700 1 --2- 172.21.15.17:0/1815092448 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7abc100dd0 0x7f7abc101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:01.193 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.211+0000 7f7ac47ff700 1 -- 172.21.15.17:0/1815092448 >> 172.21.15.17:0/1815092448 conn(0x7f7abc0fb200 msgr2=0x7f7abc0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:01.194 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.211+0000 7f7ac47ff700 1 -- 172.21.15.17:0/1815092448 shutdown_connections 2023-07-14T12:00:01.194 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.211+0000 7f7ac47ff700 1 -- 172.21.15.17:0/1815092448 wait complete. 2023-07-14T12:00:01.195 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.212+0000 7f7ac47ff700 1 Processor -- start 2023-07-14T12:00:01.195 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.213+0000 7f7ac47ff700 1 -- start start 2023-07-14T12:00:01.195 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.213+0000 7f7ac47ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7abc0ffbe0 0x7f7abc197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:01.196 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.213+0000 7f7ac47ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7abc100dd0 0x7f7abc198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:01.196 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.213+0000 7f7ac47ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7abc101760 0x7f7abc19f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:01.196 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.213+0000 7f7ac47ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7abc19f6d0 con 0x7f7abc0ffbe0 2023-07-14T12:00:01.196 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.213+0000 7f7ac47ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7abc19f810 con 0x7f7abc100dd0 2023-07-14T12:00:01.196 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.213+0000 7f7ac47ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7abc19f9a0 con 0x7f7abc101760 2023-07-14T12:00:01.196 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.213+0000 7f7ac259b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7abc0ffbe0 0x7f7abc197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:01.196 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.213+0000 7f7ac259b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7abc0ffbe0 0x7f7abc197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34704/0 (socket says 172.21.15.17:34704) 2023-07-14T12:00:01.197 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.213+0000 7f7ac259b700 1 -- 172.21.15.17:0/3116384593 learned_addr learned my addr 172.21.15.17:0/3116384593 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:00:01.197 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.213+0000 7f7ac2d9c700 1 --2- 172.21.15.17:0/3116384593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7abc101760 0x7f7abc19f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:01.197 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.213+0000 7f7ac1d9a700 1 --2- 172.21.15.17:0/3116384593 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7abc100dd0 0x7f7abc198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:01.197 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.214+0000 7f7ac259b700 1 -- 172.21.15.17:0/3116384593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7abc101760 msgr2=0x7f7abc19f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:01.197 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.214+0000 7f7ac259b700 1 --2- 172.21.15.17:0/3116384593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7abc101760 0x7f7abc19f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:01.197 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.214+0000 7f7ac259b700 1 -- 172.21.15.17:0/3116384593 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7abc100dd0 msgr2=0x7f7abc198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:01.197 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.214+0000 7f7ac259b700 1 --2- 172.21.15.17:0/3116384593 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7abc100dd0 0x7f7abc198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:01.197 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.214+0000 7f7ac259b700 1 -- 172.21.15.17:0/3116384593 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab80095f0 con 0x7f7abc0ffbe0 2023-07-14T12:00:01.198 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.214+0000 7f7ac259b700 1 --2- 172.21.15.17:0/3116384593 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7abc0ffbe0 0x7f7abc197b50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8005740 tx=0x7f7ab8005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:01.198 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.214+0000 7f7ab37fe700 1 -- 172.21.15.17:0/3116384593 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab801d070 con 0x7f7abc0ffbe0 2023-07-14T12:00:01.198 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.214+0000 7f7ab37fe700 1 -- 172.21.15.17:0/3116384593 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ab8021410 con 0x7f7abc0ffbe0 2023-07-14T12:00:01.198 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.214+0000 7f7ac47ff700 1 -- 172.21.15.17:0/3116384593 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7abc19fc20 con 0x7f7abc0ffbe0 2023-07-14T12:00:01.198 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.214+0000 7f7ab37fe700 1 -- 172.21.15.17:0/3116384593 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8017650 con 0x7f7abc0ffbe0 2023-07-14T12:00:01.198 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.214+0000 7f7ac47ff700 1 -- 172.21.15.17:0/3116384593 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7abc1a0040 con 0x7f7abc0ffbe0 2023-07-14T12:00:01.199 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.216+0000 7f7ac47ff700 1 -- 172.21.15.17:0/3116384593 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7abc10b400 con 0x7f7abc0ffbe0 2023-07-14T12:00:01.199 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.218+0000 7f7ab37fe700 1 -- 172.21.15.17:0/3116384593 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7ab8021580 con 0x7f7abc0ffbe0 2023-07-14T12:00:01.200 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.219+0000 7f7ab37fe700 1 --2- 172.21.15.17:0/3116384593 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f7aa80a1150 0x7f7aa80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:01.200 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.219+0000 7f7ab37fe700 1 -- 172.21.15.17:0/3116384593 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f7ab80ce120 con 0x7f7abc0ffbe0 2023-07-14T12:00:01.200 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.219+0000 7f7ac1d9a700 1 --2- 172.21.15.17:0/3116384593 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f7aa80a1150 0x7f7aa80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:01.201 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.220+0000 7f7ac1d9a700 1 --2- 172.21.15.17:0/3116384593 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f7aa80a1150 0x7f7aa80a3600 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7aac009c60 tx=0x7f7aac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:01.201 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.221+0000 7f7ab37fe700 1 -- 172.21.15.17:0/3116384593 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7ab80976b0 con 0x7f7abc0ffbe0 2023-07-14T12:00:01.411 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.431+0000 7f7ac47ff700 1 -- 172.21.15.17:0/3116384593 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7abc1a05f0 con 0x7f7abc0ffbe0 2023-07-14T12:00:01.412 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.431+0000 7f7ab37fe700 1 -- 172.21.15.17:0/3116384593 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7ab809acd0 con 0x7f7abc0ffbe0 2023-07-14T12:00:01.412 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2023-07-14T12:00:01.414 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.434+0000 7f7ac47ff700 1 -- 172.21.15.17:0/3116384593 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f7aa80a1150 msgr2=0x7f7aa80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:01.414 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.434+0000 7f7ac47ff700 1 --2- 172.21.15.17:0/3116384593 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f7aa80a1150 0x7f7aa80a3600 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7aac009c60 tx=0x7f7aac009260 comp rx=0 tx=0).stop 2023-07-14T12:00:01.415 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.434+0000 7f7ac47ff700 1 -- 172.21.15.17:0/3116384593 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7abc0ffbe0 msgr2=0x7f7abc197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:01.415 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.434+0000 7f7ac47ff700 1 --2- 172.21.15.17:0/3116384593 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7abc0ffbe0 0x7f7abc197b50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8005740 tx=0x7f7ab8005770 comp rx=0 tx=0).stop 2023-07-14T12:00:01.415 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.434+0000 7f7ac47ff700 1 -- 172.21.15.17:0/3116384593 shutdown_connections 2023-07-14T12:00:01.415 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.434+0000 7f7ac47ff700 1 --2- 172.21.15.17:0/3116384593 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f7aa80a1150 0x7f7aa80a3600 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:01.415 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.434+0000 7f7ac47ff700 1 --2- 172.21.15.17:0/3116384593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7abc101760 0x7f7abc19f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:01.415 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.434+0000 7f7ac47ff700 1 --2- 172.21.15.17:0/3116384593 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7abc0ffbe0 0x7f7abc197b50 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:01.415 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.434+0000 7f7ac47ff700 1 --2- 172.21.15.17:0/3116384593 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7abc100dd0 0x7f7abc198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:01.416 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.434+0000 7f7ac47ff700 1 -- 172.21.15.17:0/3116384593 >> 172.21.15.17:0/3116384593 conn(0x7f7abc0fb200 msgr2=0x7f7abc0fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:01.416 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.434+0000 7f7ac47ff700 1 -- 172.21.15.17:0/3116384593 shutdown_connections 2023-07-14T12:00:01.416 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.434+0000 7f7ac47ff700 1 -- 172.21.15.17:0/3116384593 wait complete. 2023-07-14T12:00:01.432 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-14T12:00:01.638 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.657+0000 7feb105ff700 1 -- 172.21.15.17:0/814521002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb08072e60 msgr2=0x7feb080732d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:01.638 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.657+0000 7feb105ff700 1 --2- 172.21.15.17:0/814521002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb08072e60 0x7feb080732d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7feaf8009990 tx=0x7feaf8009ca0 comp rx=0 tx=0).stop 2023-07-14T12:00:01.638 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.657+0000 7feb105ff700 1 -- 172.21.15.17:0/814521002 shutdown_connections 2023-07-14T12:00:01.638 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.657+0000 7feb105ff700 1 --2- 172.21.15.17:0/814521002 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb08074430 0x7feb08072920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:01.639 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.657+0000 7feb105ff700 1 --2- 172.21.15.17:0/814521002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb08107ac0 0x7feb08109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:01.639 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.657+0000 7feb105ff700 1 --2- 172.21.15.17:0/814521002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb08072e60 0x7feb080732d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:01.639 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.657+0000 7feb105ff700 1 -- 172.21.15.17:0/814521002 >> 172.21.15.17:0/814521002 conn(0x7feb080fb200 msgr2=0x7feb080fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:01.639 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.657+0000 7feb105ff700 1 -- 172.21.15.17:0/814521002 shutdown_connections 2023-07-14T12:00:01.639 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.657+0000 7feb105ff700 1 -- 172.21.15.17:0/814521002 wait complete. 2023-07-14T12:00:01.641 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.658+0000 7feb105ff700 1 Processor -- start 2023-07-14T12:00:01.641 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.658+0000 7feb105ff700 1 -- start start 2023-07-14T12:00:01.641 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.659+0000 7feb105ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb08072e60 0x7feb08197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:01.641 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.659+0000 7feb105ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb08074430 0x7feb08197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:01.641 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.659+0000 7feb105ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb08107ac0 0x7feb0819f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:01.641 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.659+0000 7feb105ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0819f620 con 0x7feb08107ac0 2023-07-14T12:00:01.642 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.659+0000 7feb105ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0819f760 con 0x7feb08072e60 2023-07-14T12:00:01.642 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.659+0000 7feb105ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0819f8f0 con 0x7feb08074430 2023-07-14T12:00:01.642 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.659+0000 7feb0eb9c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb08107ac0 0x7feb0819f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:01.642 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.659+0000 7feb0e39b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb08072e60 0x7feb08197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:01.642 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.659+0000 7feb0eb9c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb08107ac0 0x7feb0819f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34710/0 (socket says 172.21.15.17:34710) 2023-07-14T12:00:01.642 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.659+0000 7feb0e39b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb08072e60 0x7feb08197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.17:40230/0 (socket says 172.21.15.17:40230) 2023-07-14T12:00:01.642 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.659+0000 7feb0eb9c700 1 -- 172.21.15.17:0/1229126867 learned_addr learned my addr 172.21.15.17:0/1229126867 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:00:01.642 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.659+0000 7feb0db9a700 1 --2- 172.21.15.17:0/1229126867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb08074430 0x7feb08197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:01.643 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.659+0000 7feb0e39b700 1 -- 172.21.15.17:0/1229126867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb08074430 msgr2=0x7feb08197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:01.643 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.659+0000 7feb0e39b700 1 --2- 172.21.15.17:0/1229126867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb08074430 0x7feb08197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:01.643 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.659+0000 7feb0e39b700 1 -- 172.21.15.17:0/1229126867 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb08107ac0 msgr2=0x7feb0819f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:01.643 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.659+0000 7feb0e39b700 1 --2- 172.21.15.17:0/1229126867 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb08107ac0 0x7feb0819f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:01.643 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.659+0000 7feb0e39b700 1 -- 172.21.15.17:0/1229126867 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf80095f0 con 0x7feb08072e60 2023-07-14T12:00:01.643 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.659+0000 7feb0eb9c700 1 --2- 172.21.15.17:0/1229126867 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb08107ac0 0x7feb0819f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T12:00:01.643 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.660+0000 7feb0e39b700 1 --2- 172.21.15.17:0/1229126867 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb08072e60 0x7feb08197aa0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7feb0400e9c0 tx=0x7feb0400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:01.643 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.660+0000 7feaff7fe700 1 -- 172.21.15.17:0/1229126867 <== mon.2 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb04004220 con 0x7feb08072e60 2023-07-14T12:00:01.644 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.660+0000 7feb105ff700 1 -- 172.21.15.17:0/1229126867 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb0819fbd0 con 0x7feb08072e60 2023-07-14T12:00:01.644 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.660+0000 7feaff7fe700 1 -- 172.21.15.17:0/1229126867 <== mon.2 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb04004380 con 0x7feb08072e60 2023-07-14T12:00:01.644 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.660+0000 7feaff7fe700 1 -- 172.21.15.17:0/1229126867 <== mon.2 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb04003be0 con 0x7feb08072e60 2023-07-14T12:00:01.644 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.660+0000 7feb105ff700 1 -- 172.21.15.17:0/1229126867 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb081a0130 con 0x7feb08072e60 2023-07-14T12:00:01.645 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.661+0000 7feb105ff700 1 -- 172.21.15.17:0/1229126867 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb08106960 con 0x7feb08072e60 2023-07-14T12:00:01.645 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.663+0000 7feaff7fe700 1 -- 172.21.15.17:0/1229126867 <== mon.2 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7feb04003d40 con 0x7feb08072e60 2023-07-14T12:00:01.645 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.665+0000 7feaff7fe700 1 --2- 172.21.15.17:0/1229126867 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7feaf40a1150 0x7feaf40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:01.645 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.665+0000 7feaff7fe700 1 -- 172.21.15.17:0/1229126867 <== mon.2 v2:172.21.15.93:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7feb04014070 con 0x7feb08072e60 2023-07-14T12:00:01.646 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.665+0000 7feb0db9a700 1 --2- 172.21.15.17:0/1229126867 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7feaf40a1150 0x7feaf40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:01.646 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.666+0000 7feb0db9a700 1 --2- 172.21.15.17:0/1229126867 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7feaf40a1150 0x7feaf40a3600 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7feaf800b680 tx=0x7feaf8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:01.647 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.666+0000 7feaff7fe700 1 -- 172.21.15.17:0/1229126867 <== mon.2 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7feb04096460 con 0x7feb08072e60 2023-07-14T12:00:01.857 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.877+0000 7feb105ff700 1 -- 172.21.15.17:0/1229126867 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7feb0802cac0 con 0x7feb08072e60 2023-07-14T12:00:01.858 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.877+0000 7feaff7fe700 1 -- 172.21.15.17:0/1229126867 <== mon.2 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7feb04099a80 con 0x7feb08072e60 2023-07-14T12:00:01.858 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2023-07-14T12:00:01.861 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.880+0000 7feb105ff700 1 -- 172.21.15.17:0/1229126867 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7feaf40a1150 msgr2=0x7feaf40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:01.861 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.880+0000 7feb105ff700 1 --2- 172.21.15.17:0/1229126867 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7feaf40a1150 0x7feaf40a3600 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7feaf800b680 tx=0x7feaf8005fd0 comp rx=0 tx=0).stop 2023-07-14T12:00:01.861 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.880+0000 7feb105ff700 1 -- 172.21.15.17:0/1229126867 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb08072e60 msgr2=0x7feb08197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:01.861 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.880+0000 7feb105ff700 1 --2- 172.21.15.17:0/1229126867 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb08072e60 0x7feb08197aa0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7feb0400e9c0 tx=0x7feb0400ecd0 comp rx=0 tx=0).stop 2023-07-14T12:00:01.861 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.880+0000 7feb105ff700 1 -- 172.21.15.17:0/1229126867 shutdown_connections 2023-07-14T12:00:01.861 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.880+0000 7feb105ff700 1 --2- 172.21.15.17:0/1229126867 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7feaf40a1150 0x7feaf40a3600 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:01.862 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.880+0000 7feb105ff700 1 --2- 172.21.15.17:0/1229126867 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb08072e60 0x7feb08197aa0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:01.862 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.880+0000 7feb105ff700 1 --2- 172.21.15.17:0/1229126867 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb08107ac0 0x7feb0819f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:01.862 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.880+0000 7feb105ff700 1 --2- 172.21.15.17:0/1229126867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb08074430 0x7feb08197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:01.862 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.880+0000 7feb105ff700 1 -- 172.21.15.17:0/1229126867 >> 172.21.15.17:0/1229126867 conn(0x7feb080fb200 msgr2=0x7feb08103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:01.862 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.880+0000 7feb105ff700 1 -- 172.21.15.17:0/1229126867 shutdown_connections 2023-07-14T12:00:01.862 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:01.880+0000 7feb105ff700 1 -- 172.21.15.17:0/1229126867 wait complete. 2023-07-14T12:00:01.879 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-14T12:00:02.084 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.103+0000 7f00a77ff700 1 -- 172.21.15.17:0/4112166492 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00a0101760 msgr2=0x7f00a0108020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:02.084 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.103+0000 7f00a77ff700 1 --2- 172.21.15.17:0/4112166492 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00a0101760 0x7f00a0108020 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f009800b0a0 tx=0x7f009800b3b0 comp rx=0 tx=0).stop 2023-07-14T12:00:02.085 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.103+0000 7f00a77ff700 1 -- 172.21.15.17:0/4112166492 shutdown_connections 2023-07-14T12:00:02.085 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.103+0000 7f00a77ff700 1 --2- 172.21.15.17:0/4112166492 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00a0101760 0x7f00a0108020 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:02.085 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.103+0000 7f00a77ff700 1 --2- 172.21.15.17:0/4112166492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00a00ffbe0 0x7f00a00ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:02.085 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.103+0000 7f00a77ff700 1 --2- 172.21.15.17:0/4112166492 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f00a0100dd0 0x7f00a0101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:02.085 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.103+0000 7f00a77ff700 1 -- 172.21.15.17:0/4112166492 >> 172.21.15.17:0/4112166492 conn(0x7f00a00fb200 msgr2=0x7f00a00fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:02.085 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.103+0000 7f00a77ff700 1 -- 172.21.15.17:0/4112166492 shutdown_connections 2023-07-14T12:00:02.085 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.103+0000 7f00a77ff700 1 -- 172.21.15.17:0/4112166492 wait complete. 2023-07-14T12:00:02.087 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.104+0000 7f00a77ff700 1 Processor -- start 2023-07-14T12:00:02.088 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.105+0000 7f00a77ff700 1 -- start start 2023-07-14T12:00:02.088 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.105+0000 7f00a77ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00a00ffbe0 0x7f00a0197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:02.088 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.105+0000 7f00a77ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f00a0100dd0 0x7f00a0198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:02.089 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.105+0000 7f00a77ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00a0101760 0x7f00a019f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:02.089 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.105+0000 7f00a77ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a019f740 con 0x7f00a0101760 2023-07-14T12:00:02.089 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.105+0000 7f00a77ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a019f880 con 0x7f00a0100dd0 2023-07-14T12:00:02.090 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.105+0000 7f00a77ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a019fa10 con 0x7f00a00ffbe0 2023-07-14T12:00:02.090 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.105+0000 7f00a5d9c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00a0101760 0x7f00a019f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:02.090 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.105+0000 7f00a5d9c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00a0101760 0x7f00a019f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34732/0 (socket says 172.21.15.17:34732) 2023-07-14T12:00:02.091 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.105+0000 7f00a5d9c700 1 -- 172.21.15.17:0/2412546168 learned_addr learned my addr 172.21.15.17:0/2412546168 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:00:02.091 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.105+0000 7f00a559b700 1 --2- 172.21.15.17:0/2412546168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00a00ffbe0 0x7f00a0197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:02.091 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.105+0000 7f00a4d9a700 1 --2- 172.21.15.17:0/2412546168 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f00a0100dd0 0x7f00a0198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:02.092 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.106+0000 7f00a5d9c700 1 -- 172.21.15.17:0/2412546168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00a00ffbe0 msgr2=0x7f00a0197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:02.092 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.106+0000 7f00a5d9c700 1 --2- 172.21.15.17:0/2412546168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00a00ffbe0 0x7f00a0197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:02.092 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.106+0000 7f00a5d9c700 1 -- 172.21.15.17:0/2412546168 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f00a0100dd0 msgr2=0x7f00a0198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:02.092 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.106+0000 7f00a5d9c700 1 --2- 172.21.15.17:0/2412546168 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f00a0100dd0 0x7f00a0198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:02.093 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.106+0000 7f00a5d9c700 1 -- 172.21.15.17:0/2412546168 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0098009d00 con 0x7f00a0101760 2023-07-14T12:00:02.093 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.106+0000 7f00a559b700 1 --2- 172.21.15.17:0/2412546168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00a00ffbe0 0x7f00a0197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T12:00:02.093 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.106+0000 7f00a5d9c700 1 --2- 172.21.15.17:0/2412546168 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00a0101760 0x7f00a019f170 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f009800b780 tx=0x7f009800bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:02.094 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.106+0000 7f00967fc700 1 -- 172.21.15.17:0/2412546168 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009800e040 con 0x7f00a0101760 2023-07-14T12:00:02.094 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.106+0000 7f00967fc700 1 -- 172.21.15.17:0/2412546168 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0098004e60 con 0x7f00a0101760 2023-07-14T12:00:02.094 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.106+0000 7f00a77ff700 1 -- 172.21.15.17:0/2412546168 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00a019fc90 con 0x7f00a0101760 2023-07-14T12:00:02.094 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.106+0000 7f00967fc700 1 -- 172.21.15.17:0/2412546168 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009801b740 con 0x7f00a0101760 2023-07-14T12:00:02.095 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.106+0000 7f00a77ff700 1 -- 172.21.15.17:0/2412546168 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00a01a0160 con 0x7f00a0101760 2023-07-14T12:00:02.097 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.107+0000 7f00a77ff700 1 -- 172.21.15.17:0/2412546168 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00a010b3c0 con 0x7f00a0101760 2023-07-14T12:00:02.097 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.109+0000 7f00967fc700 1 -- 172.21.15.17:0/2412546168 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f009801da40 con 0x7f00a0101760 2023-07-14T12:00:02.097 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.111+0000 7f00967fc700 1 --2- 172.21.15.17:0/2412546168 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f008c0a11d0 0x7f008c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:02.097 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.111+0000 7f00967fc700 1 -- 172.21.15.17:0/2412546168 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f00980ce260 con 0x7f00a0101760 2023-07-14T12:00:02.098 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.111+0000 7f00a559b700 1 --2- 172.21.15.17:0/2412546168 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f008c0a11d0 0x7f008c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:02.098 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.112+0000 7f00a559b700 1 --2- 172.21.15.17:0/2412546168 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f008c0a11d0 0x7f008c0a3680 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f00a0100c30 tx=0x7f009c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:02.098 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.114+0000 7f00967fc700 1 -- 172.21.15.17:0/2412546168 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f00980977f0 con 0x7f00a0101760 2023-07-14T12:00:02.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:01 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/3759411032' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T12:00:02.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:01 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/3116384593' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T12:00:02.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:01 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/3759411032' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T12:00:02.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:01 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/3116384593' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T12:00:02.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:01 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/3759411032' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T12:00:02.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:01 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/3116384593' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T12:00:02.322 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.342+0000 7f00a77ff700 1 -- 172.21.15.17:0/2412546168 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f00a010b5b0 con 0x7f00a0101760 2023-07-14T12:00:02.323 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.342+0000 7f00967fc700 1 -- 172.21.15.17:0/2412546168 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) v1 ==== 81+0+592 (secure 0 0 0) 0x7f009809ae10 con 0x7f00a0101760 2023-07-14T12:00:02.323 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/2084033276 2023-07-15T11:53:05.639597+0000 2023-07-14T12:00:02.323 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:6801/1059284716 2023-07-15T11:53:05.639597+0000 2023-07-14T12:00:02.323 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/3651785636 2023-07-15T11:53:05.639597+0000 2023-07-14T12:00:02.324 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/1373538024 2023-07-15T11:52:37.887946+0000 2023-07-14T12:00:02.324 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/3560395020 2023-07-15T11:52:37.887946+0000 2023-07-14T12:00:02.324 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:6800/1059284716 2023-07-15T11:53:05.639597+0000 2023-07-14T12:00:02.324 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/4262912861 2023-07-15T11:53:05.639597+0000 2023-07-14T12:00:02.324 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/2258375202 2023-07-15T11:52:37.887946+0000 2023-07-14T12:00:02.324 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:6800/2804227742 2023-07-15T11:52:37.887946+0000 2023-07-14T12:00:02.324 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:6801/2804227742 2023-07-15T11:52:37.887946+0000 2023-07-14T12:00:02.325 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.344+0000 7f00a77ff700 1 -- 172.21.15.17:0/2412546168 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f008c0a11d0 msgr2=0x7f008c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:02.325 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.344+0000 7f00a77ff700 1 --2- 172.21.15.17:0/2412546168 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f008c0a11d0 0x7f008c0a3680 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f00a0100c30 tx=0x7f009c009260 comp rx=0 tx=0).stop 2023-07-14T12:00:02.326 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.344+0000 7f00a77ff700 1 -- 172.21.15.17:0/2412546168 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00a0101760 msgr2=0x7f00a019f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:02.326 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.344+0000 7f00a77ff700 1 --2- 172.21.15.17:0/2412546168 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00a0101760 0x7f00a019f170 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f009800b780 tx=0x7f009800bd00 comp rx=0 tx=0).stop 2023-07-14T12:00:02.326 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.344+0000 7f00a77ff700 1 -- 172.21.15.17:0/2412546168 shutdown_connections 2023-07-14T12:00:02.326 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.344+0000 7f00a77ff700 1 --2- 172.21.15.17:0/2412546168 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f008c0a11d0 0x7f008c0a3680 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:02.326 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.344+0000 7f00a77ff700 1 --2- 172.21.15.17:0/2412546168 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00a0101760 0x7f00a019f170 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:02.326 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.344+0000 7f00a77ff700 1 --2- 172.21.15.17:0/2412546168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00a00ffbe0 0x7f00a0197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:02.326 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.344+0000 7f00a77ff700 1 --2- 172.21.15.17:0/2412546168 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f00a0100dd0 0x7f00a0198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:02.327 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.344+0000 7f00a77ff700 1 -- 172.21.15.17:0/2412546168 >> 172.21.15.17:0/2412546168 conn(0x7f00a00fb200 msgr2=0x7f00a0106620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:02.327 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.344+0000 7f00a77ff700 1 -- 172.21.15.17:0/2412546168 shutdown_connections 2023-07-14T12:00:02.327 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.345+0000 7f00a77ff700 1 -- 172.21.15.17:0/2412546168 wait complete. 2023-07-14T12:00:02.327 INFO:teuthology.orchestra.run.smithi017.stderr:listed 10 entries 2023-07-14T12:00:02.343 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2023-07-14T12:00:02.343 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T12:00:02.387 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T12:00:02.630 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.649+0000 7f252ffff700 1 -- 172.21.15.17:0/3622731034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25280ffbb0 msgr2=0x7f25280fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:02.630 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.649+0000 7f252ffff700 1 --2- 172.21.15.17:0/3622731034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25280ffbb0 0x7f25280fffc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2524009990 tx=0x7f2524009ca0 comp rx=0 tx=0).stop 2023-07-14T12:00:02.630 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.649+0000 7f252ffff700 1 -- 172.21.15.17:0/3622731034 shutdown_connections 2023-07-14T12:00:02.631 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.649+0000 7f252ffff700 1 --2- 172.21.15.17:0/3622731034 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25281017f0 0x7f2528108090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:02.631 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.649+0000 7f252ffff700 1 --2- 172.21.15.17:0/3622731034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25280ffbb0 0x7f25280fffc0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:02.631 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.649+0000 7f252ffff700 1 --2- 172.21.15.17:0/3622731034 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2528100e40 0x7f25281012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:02.631 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.649+0000 7f252ffff700 1 -- 172.21.15.17:0/3622731034 >> 172.21.15.17:0/3622731034 conn(0x7f25280fb1d0 msgr2=0x7f25280fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:02.631 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.649+0000 7f252ffff700 1 -- 172.21.15.17:0/3622731034 shutdown_connections 2023-07-14T12:00:02.632 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.649+0000 7f252ffff700 1 -- 172.21.15.17:0/3622731034 wait complete. 2023-07-14T12:00:02.634 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.650+0000 7f252ffff700 1 Processor -- start 2023-07-14T12:00:02.634 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.650+0000 7f252ffff700 1 -- start start 2023-07-14T12:00:02.634 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.651+0000 7f252ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f25280ffbb0 0x7f2528197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:02.634 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.651+0000 7f252ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2528100e40 0x7f2528198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:02.635 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.651+0000 7f252ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25281017f0 0x7f252819f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:02.635 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.651+0000 7f252ffff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252819f680 con 0x7f25281017f0 2023-07-14T12:00:02.635 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.651+0000 7f252ffff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252819f7c0 con 0x7f25280ffbb0 2023-07-14T12:00:02.635 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.651+0000 7f252ffff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252819f950 con 0x7f2528100e40 2023-07-14T12:00:02.635 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.651+0000 7f252dd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f25280ffbb0 0x7f2528197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:02.635 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.651+0000 7f252d59a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2528100e40 0x7f2528198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:02.635 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.651+0000 7f252dd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f25280ffbb0 0x7f2528197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.17:40266/0 (socket says 172.21.15.17:40266) 2023-07-14T12:00:02.635 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.651+0000 7f252d59a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2528100e40 0x7f2528198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.17:35416/0 (socket says 172.21.15.17:35416) 2023-07-14T12:00:02.636 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.651+0000 7f252e59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25281017f0 0x7f252819f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:02.636 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.651+0000 7f252dd9b700 1 -- 172.21.15.17:0/2419512479 learned_addr learned my addr 172.21.15.17:0/2419512479 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:00:02.636 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.651+0000 7f252d59a700 1 -- 172.21.15.17:0/2419512479 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f25280ffbb0 msgr2=0x7f2528197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:02.636 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.651+0000 7f252d59a700 1 --2- 172.21.15.17:0/2419512479 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f25280ffbb0 0x7f2528197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:02.636 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.651+0000 7f252d59a700 1 -- 172.21.15.17:0/2419512479 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25281017f0 msgr2=0x7f252819f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:02.636 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.651+0000 7f252d59a700 1 --2- 172.21.15.17:0/2419512479 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25281017f0 0x7f252819f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:02.637 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.651+0000 7f252d59a700 1 -- 172.21.15.17:0/2419512479 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25240095f0 con 0x7f2528100e40 2023-07-14T12:00:02.637 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.652+0000 7f252d59a700 1 --2- 172.21.15.17:0/2419512479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2528100e40 0x7f2528198040 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f251800e9f0 tx=0x7f251800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:02.637 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.652+0000 7f251effd700 1 -- 172.21.15.17:0/2419512479 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251800c9c0 con 0x7f2528100e40 2023-07-14T12:00:02.637 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.652+0000 7f251effd700 1 -- 172.21.15.17:0/2419512479 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2518010430 con 0x7f2528100e40 2023-07-14T12:00:02.637 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.652+0000 7f252ffff700 1 -- 172.21.15.17:0/2419512479 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f252819fc30 con 0x7f2528100e40 2023-07-14T12:00:02.637 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.652+0000 7f251effd700 1 -- 172.21.15.17:0/2419512479 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2518018680 con 0x7f2528100e40 2023-07-14T12:00:02.637 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.652+0000 7f252ffff700 1 -- 172.21.15.17:0/2419512479 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25281a0190 con 0x7f2528100e40 2023-07-14T12:00:02.637 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.653+0000 7f252ffff700 1 -- 172.21.15.17:0/2419512479 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2528192290 con 0x7f2528100e40 2023-07-14T12:00:02.638 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.655+0000 7f251effd700 1 -- 172.21.15.17:0/2419512479 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2518010860 con 0x7f2528100e40 2023-07-14T12:00:02.638 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.657+0000 7f251effd700 1 --2- 172.21.15.17:0/2419512479 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f25140a11d0 0x7f25140a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:02.639 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.657+0000 7f251effd700 1 -- 172.21.15.17:0/2419512479 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f2518014070 con 0x7f2528100e40 2023-07-14T12:00:02.639 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.657+0000 7f252dd9b700 1 --2- 172.21.15.17:0/2419512479 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f25140a11d0 0x7f25140a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:02.639 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.657+0000 7f252dd9b700 1 --2- 172.21.15.17:0/2419512479 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f25140a11d0 0x7f25140a3680 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f252400b2e0 tx=0x7f2524005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:02.639 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.658+0000 7f251effd700 1 -- 172.21.15.17:0/2419512479 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f25180962d0 con 0x7f2528100e40 2023-07-14T12:00:02.852 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.871+0000 7f252ffff700 1 -- 172.21.15.17:0/2419512479 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f252802c850 con 0x7f2528100e40 2023-07-14T12:00:02.853 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.874+0000 7f251effd700 1 -- 172.21.15.17:0/2419512479 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23866 (secure 0 0 0) 0x7f2518026090 con 0x7f2528100e40 2023-07-14T12:00:02.854 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T12:00:02.854 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":88,"fsid":"b3c386cc-223c-11ee-9b33-001a4aab830c","created":"2023-07-14T11:51:59.992960+0000","modified":"2023-07-14T11:59:48.295704+0000","last_up_change":"2023-07-14T11:58:55.037769+0000","last_in_change":"2023-07-14T11:58:37.114870+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:55:28.237002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:59:41.560502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T11:59:42.725461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:59:46.267029+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"57003438-a83c-4c8e-a029-56414740895c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6803","nonce":3588366384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6805","nonce":3588366384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6809","nonce":3588366384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6807","nonce":3588366384}]},"public_addr":"172.21.15.17:6803/3588366384","cluster_addr":"172.21.15.17:6805/3588366384","heartbeat_back_addr":"172.21.15.17:6809/3588366384","heartbeat_front_addr":"172.21.15.17:6807/3588366384","state":["exists","up"]},{"osd":1,"uuid":"f56b6337-d8a9-4411-b9d8-b6863dc98556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6801","nonce":3147203064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6803","nonce":3147203064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6807","nonce":3147203064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6805","nonce":3147203064}]},"public_addr":"172.21.15.93:6801/3147203064","cluster_addr":"172.21.15.93:6803/3147203064","heartbeat_back_addr":"172.21.15.93:6807/3147203064","heartbeat_front_addr":"172.21.15.93:6805/3147203064","state":["exists","up"]},{"osd":2,"uuid":"fc34bb7f-551f-46e1-99c0-59924d6c1346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6801","nonce":1877269815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6803","nonce":1877269815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6807","nonce":1877269815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6805","nonce":1877269815}]},"public_addr":"172.21.15.97:6801/1877269815","cluster_addr":"172.21.15.97:6803/1877269815","heartbeat_back_addr":"172.21.15.97:6807/1877269815","heartbeat_front_addr":"172.21.15.97:6805/1877269815","state":["exists","up"]},{"osd":3,"uuid":"37a1583c-4fb8-412a-a0c8-f83b1ff26a91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6811","nonce":2784619821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6813","nonce":2784619821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6817","nonce":2784619821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6815","nonce":2784619821}]},"public_addr":"172.21.15.17:6811/2784619821","cluster_addr":"172.21.15.17:6813/2784619821","heartbeat_back_addr":"172.21.15.17:6817/2784619821","heartbeat_front_addr":"172.21.15.17:6815/2784619821","state":["exists","up"]},{"osd":4,"uuid":"f4b9f377-d519-4b14-ac4a-f1debaeed9f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6809","nonce":3151801173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6811","nonce":3151801173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6815","nonce":3151801173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6813","nonce":3151801173}]},"public_addr":"172.21.15.93:6809/3151801173","cluster_addr":"172.21.15.93:6811/3151801173","heartbeat_back_addr":"172.21.15.93:6815/3151801173","heartbeat_front_addr":"172.21.15.93:6813/3151801173","state":["exists","up"]},{"osd":5,"uuid":"2d2af354-0535-4ef9-b166-3ef219498db8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6809","nonce":2184413924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6811","nonce":2184413924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6815","nonce":2184413924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6813","nonce":2184413924}]},"public_addr":"172.21.15.97:6809/2184413924","cluster_addr":"172.21.15.97:6811/2184413924","heartbeat_back_addr":"172.21.15.97:6815/2184413924","heartbeat_front_addr":"172.21.15.97:6813/2184413924","state":["exists","up"]},{"osd":6,"uuid":"9cdf0c4b-62dc-4168-864e-a9d1650a9ea8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6819","nonce":50486581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6821","nonce":50486581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6825","nonce":50486581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6823","nonce":50486581}]},"public_addr":"172.21.15.17:6819/50486581","cluster_addr":"172.21.15.17:6821/50486581","heartbeat_back_addr":"172.21.15.17:6825/50486581","heartbeat_front_addr":"172.21.15.17:6823/50486581","state":["exists","up"]},{"osd":7,"uuid":"09f60823-5491-45cd-9324-6ae0d56ce262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6817","nonce":2596912392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6819","nonce":2596912392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6823","nonce":2596912392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6821","nonce":2596912392}]},"public_addr":"172.21.15.93:6817/2596912392","cluster_addr":"172.21.15.93:6819/2596912392","heartbeat_back_addr":"172.21.15.93:6823/2596912392","heartbeat_front_addr":"172.21.15.93:6821/2596912392","state":["exists","up"]},{"osd":8,"uuid":"8c5771c9-718e-4313-a2b6-5e1c37c53f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6817","nonce":1965492859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6819","nonce":1965492859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6823","nonce":1965492859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6821","nonce":1965492859}]},"public_addr":"172.21.15.97:6817/1965492859","cluster_addr":"172.21.15.97:6819/1965492859","heartbeat_back_addr":"172.21.15.97:6823/1965492859","heartbeat_front_addr":"172.21.15.97:6821/1965492859","state":["exists","up"]},{"osd":9,"uuid":"1197b8a3-c09f-4c57-aba1-9e988a1620b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6827","nonce":2512645700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6829","nonce":2512645700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6833","nonce":2512645700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6831","nonce":2512645700}]},"public_addr":"172.21.15.17:6827/2512645700","cluster_addr":"172.21.15.17:6829/2512645700","heartbeat_back_addr":"172.21.15.17:6833/2512645700","heartbeat_front_addr":"172.21.15.17:6831/2512645700","state":["exists","up"]},{"osd":10,"uuid":"f99918c2-cdd8-4a7a-be2e-d1d6e0f828a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6825","nonce":264429558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6827","nonce":264429558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6831","nonce":264429558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6829","nonce":264429558}]},"public_addr":"172.21.15.93:6825/264429558","cluster_addr":"172.21.15.93:6827/264429558","heartbeat_back_addr":"172.21.15.93:6831/264429558","heartbeat_front_addr":"172.21.15.93:6829/264429558","state":["exists","up"]},{"osd":11,"uuid":"6bbb5b79-8d4d-441f-9583-203df6c78cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6825","nonce":3389913754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6827","nonce":3389913754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6831","nonce":3389913754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6829","nonce":3389913754}]},"public_addr":"172.21.15.97:6825/3389913754","cluster_addr":"172.21.15.97:6827/3389913754","heartbeat_back_addr":"172.21.15.97:6831/3389913754","heartbeat_front_addr":"172.21.15.97:6829/3389913754","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:54:41.079618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:03.207899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:24.136961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:50.392609+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:56:12.136237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:56:33.212314+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:00.243646+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:22.059234+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:43.232399+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:10.043896+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:31.910780+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:53.158983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/2084033276":"2023-07-15T11:53:05.639597+0000","172.21.15.17:6801/1059284716":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/3651785636":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/1373538024":"2023-07-15T11:52:37.887946+0000","172.21.15.17:0/3560395020":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6800/1059284716":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/4262912861":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/2258375202":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6800/2804227742":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6801/2804227742":"2023-07-15T11:52:37.887946+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T12:00:02.856 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.876+0000 7f252ffff700 1 -- 172.21.15.17:0/2419512479 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f25140a11d0 msgr2=0x7f25140a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:02.856 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.876+0000 7f252ffff700 1 --2- 172.21.15.17:0/2419512479 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f25140a11d0 0x7f25140a3680 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f252400b2e0 tx=0x7f2524005fb0 comp rx=0 tx=0).stop 2023-07-14T12:00:02.857 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.876+0000 7f252ffff700 1 -- 172.21.15.17:0/2419512479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2528100e40 msgr2=0x7f2528198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:02.857 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.876+0000 7f252ffff700 1 --2- 172.21.15.17:0/2419512479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2528100e40 0x7f2528198040 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f251800e9f0 tx=0x7f251800ed00 comp rx=0 tx=0).stop 2023-07-14T12:00:02.857 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.876+0000 7f252ffff700 1 -- 172.21.15.17:0/2419512479 shutdown_connections 2023-07-14T12:00:02.857 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.876+0000 7f252ffff700 1 --2- 172.21.15.17:0/2419512479 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f25140a11d0 0x7f25140a3680 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:02.857 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.876+0000 7f252ffff700 1 --2- 172.21.15.17:0/2419512479 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f25280ffbb0 0x7f2528197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:02.857 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.876+0000 7f252ffff700 1 --2- 172.21.15.17:0/2419512479 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25281017f0 0x7f252819f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:02.857 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.876+0000 7f252ffff700 1 --2- 172.21.15.17:0/2419512479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2528100e40 0x7f2528198040 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:02.858 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.876+0000 7f252ffff700 1 -- 172.21.15.17:0/2419512479 >> 172.21.15.17:0/2419512479 conn(0x7f25280fb1d0 msgr2=0x7f2528106720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:02.858 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.876+0000 7f252ffff700 1 -- 172.21.15.17:0/2419512479 shutdown_connections 2023-07-14T12:00:02.858 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:02.876+0000 7f252ffff700 1 -- 172.21.15.17:0/2419512479 wait complete. 2023-07-14T12:00:02.875 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:55:28.237002+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T11:59:41.560502+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T11:59:42.725461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-14T11:59:46.267029+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-14T12:00:02.876 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T12:00:03.082 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.101+0000 7f9ce71ff700 1 -- 172.21.15.17:0/1853048680 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ce0100dd0 msgr2=0x7f9ce0101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:03.082 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.101+0000 7f9ce71ff700 1 --2- 172.21.15.17:0/1853048680 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ce0100dd0 0x7f9ce0101220 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0009990 tx=0x7f9cd0009ca0 comp rx=0 tx=0).stop 2023-07-14T12:00:03.082 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.101+0000 7f9ce71ff700 1 -- 172.21.15.17:0/1853048680 shutdown_connections 2023-07-14T12:00:03.083 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.101+0000 7f9ce71ff700 1 --2- 172.21.15.17:0/1853048680 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ce00ffbe0 0x7f9ce00ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:03.083 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.101+0000 7f9ce71ff700 1 --2- 172.21.15.17:0/1853048680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ce0101760 0x7f9ce0108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:03.083 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.101+0000 7f9ce71ff700 1 --2- 172.21.15.17:0/1853048680 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ce0100dd0 0x7f9ce0101220 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:03.083 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.101+0000 7f9ce71ff700 1 -- 172.21.15.17:0/1853048680 >> 172.21.15.17:0/1853048680 conn(0x7f9ce00fb200 msgr2=0x7f9ce00fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:03.083 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.101+0000 7f9ce71ff700 1 -- 172.21.15.17:0/1853048680 shutdown_connections 2023-07-14T12:00:03.083 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.101+0000 7f9ce71ff700 1 -- 172.21.15.17:0/1853048680 wait complete. 2023-07-14T12:00:03.085 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.102+0000 7f9ce71ff700 1 Processor -- start 2023-07-14T12:00:03.085 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.103+0000 7f9ce71ff700 1 -- start start 2023-07-14T12:00:03.085 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.103+0000 7f9ce71ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ce00ffbe0 0x7f9ce0197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:03.085 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.103+0000 7f9ce71ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ce0100dd0 0x7f9ce0198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:03.086 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.103+0000 7f9ce71ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ce0101760 0x7f9ce019f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:03.086 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.103+0000 7f9ce71ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce019f6c0 con 0x7f9ce0101760 2023-07-14T12:00:03.086 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.103+0000 7f9ce71ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce019f800 con 0x7f9ce00ffbe0 2023-07-14T12:00:03.086 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.103+0000 7f9ce71ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce019f990 con 0x7f9ce0100dd0 2023-07-14T12:00:03.086 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.103+0000 7f9ce579c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ce0101760 0x7f9ce019f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:03.086 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.103+0000 7f9ce4f9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ce00ffbe0 0x7f9ce0197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:03.086 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.103+0000 7f9ce579c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ce0101760 0x7f9ce019f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38122/0 (socket says 172.21.15.17:38122) 2023-07-14T12:00:03.086 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.103+0000 7f9ce579c700 1 -- 172.21.15.17:0/1299751110 learned_addr learned my addr 172.21.15.17:0/1299751110 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:00:03.087 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.103+0000 7f9cd7fff700 1 --2- 172.21.15.17:0/1299751110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ce0100dd0 0x7f9ce0198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:03.087 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.104+0000 7f9ce4f9b700 1 -- 172.21.15.17:0/1299751110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ce0100dd0 msgr2=0x7f9ce0198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:03.087 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.104+0000 7f9ce4f9b700 1 --2- 172.21.15.17:0/1299751110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ce0100dd0 0x7f9ce0198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:03.087 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.104+0000 7f9ce4f9b700 1 -- 172.21.15.17:0/1299751110 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ce0101760 msgr2=0x7f9ce019f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:03.087 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.104+0000 7f9ce4f9b700 1 --2- 172.21.15.17:0/1299751110 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ce0101760 0x7f9ce019f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:03.087 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.104+0000 7f9ce4f9b700 1 -- 172.21.15.17:0/1299751110 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd00095f0 con 0x7f9ce00ffbe0 2023-07-14T12:00:03.087 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.104+0000 7f9ce4f9b700 1 --2- 172.21.15.17:0/1299751110 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ce00ffbe0 0x7f9ce0197b40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc00e9c0 tx=0x7f9cdc00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:03.088 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.104+0000 7f9cd5ffb700 1 -- 172.21.15.17:0/1299751110 <== mon.2 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc004220 con 0x7f9ce00ffbe0 2023-07-14T12:00:03.088 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.104+0000 7f9cd5ffb700 1 -- 172.21.15.17:0/1299751110 <== mon.2 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cdc004380 con 0x7f9ce00ffbe0 2023-07-14T12:00:03.088 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.104+0000 7f9ce71ff700 1 -- 172.21.15.17:0/1299751110 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ce019fc70 con 0x7f9ce00ffbe0 2023-07-14T12:00:03.088 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.104+0000 7f9cd5ffb700 1 -- 172.21.15.17:0/1299751110 <== mon.2 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc003be0 con 0x7f9ce00ffbe0 2023-07-14T12:00:03.088 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.104+0000 7f9ce71ff700 1 -- 172.21.15.17:0/1299751110 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ce01a0180 con 0x7f9ce00ffbe0 2023-07-14T12:00:03.089 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.105+0000 7f9ce71ff700 1 -- 172.21.15.17:0/1299751110 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ce010b400 con 0x7f9ce00ffbe0 2023-07-14T12:00:03.089 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.107+0000 7f9cd5ffb700 1 -- 172.21.15.17:0/1299751110 <== mon.2 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9cdc003d40 con 0x7f9ce00ffbe0 2023-07-14T12:00:03.089 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.108+0000 7f9cd5ffb700 1 --2- 172.21.15.17:0/1299751110 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f9cc80a1150 0x7f9cc80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:03.089 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.110+0000 7f9cd5ffb700 1 -- 172.21.15.17:0/1299751110 <== mon.2 v2:172.21.15.93:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f9cdc014070 con 0x7f9ce00ffbe0 2023-07-14T12:00:03.090 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.110+0000 7f9cd7fff700 1 --2- 172.21.15.17:0/1299751110 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f9cc80a1150 0x7f9cc80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:03.090 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.110+0000 7f9cd7fff700 1 --2- 172.21.15.17:0/1299751110 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f9cc80a1150 0x7f9cc80a3600 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9cd000b2e0 tx=0x7f9cd0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:03.091 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.110+0000 7f9cd5ffb700 1 -- 172.21.15.17:0/1299751110 <== mon.2 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9cdc0968d0 con 0x7f9ce00ffbe0 2023-07-14T12:00:03.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:02 smithi093 ceph-mon[128922]: pgmap v286: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2023-07-14T12:00:03.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:02 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/1229126867' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T12:00:03.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:02 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/2412546168' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-14T12:00:03.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:02 smithi097 ceph-mon[129313]: pgmap v286: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2023-07-14T12:00:03.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:02 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/1229126867' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T12:00:03.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:02 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/2412546168' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-14T12:00:03.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:02 smithi017 ceph-mon[126339]: pgmap v286: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2023-07-14T12:00:03.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:02 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/1229126867' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T12:00:03.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:02 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/2412546168' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-14T12:00:03.302 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.322+0000 7f9ce71ff700 1 -- 172.21.15.17:0/1299751110 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9ce01a0730 con 0x7f9ce00ffbe0 2023-07-14T12:00:03.303 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.324+0000 7f9cd5ffb700 1 -- 172.21.15.17:0/1299751110 <== mon.2 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9cdc01f340 con 0x7f9ce00ffbe0 2023-07-14T12:00:03.304 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2023-07-14T12:00:03.306 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.325+0000 7f9ce71ff700 1 -- 172.21.15.17:0/1299751110 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f9cc80a1150 msgr2=0x7f9cc80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:03.306 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.325+0000 7f9ce71ff700 1 --2- 172.21.15.17:0/1299751110 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f9cc80a1150 0x7f9cc80a3600 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9cd000b2e0 tx=0x7f9cd0005fb0 comp rx=0 tx=0).stop 2023-07-14T12:00:03.306 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.325+0000 7f9ce71ff700 1 -- 172.21.15.17:0/1299751110 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ce00ffbe0 msgr2=0x7f9ce0197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:03.306 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.325+0000 7f9ce71ff700 1 --2- 172.21.15.17:0/1299751110 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ce00ffbe0 0x7f9ce0197b40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc00e9c0 tx=0x7f9cdc00ecd0 comp rx=0 tx=0).stop 2023-07-14T12:00:03.306 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.325+0000 7f9ce71ff700 1 -- 172.21.15.17:0/1299751110 shutdown_connections 2023-07-14T12:00:03.306 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.325+0000 7f9ce71ff700 1 --2- 172.21.15.17:0/1299751110 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f9cc80a1150 0x7f9cc80a3600 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:03.307 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.325+0000 7f9ce71ff700 1 --2- 172.21.15.17:0/1299751110 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ce00ffbe0 0x7f9ce0197b40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:03.307 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.325+0000 7f9ce71ff700 1 --2- 172.21.15.17:0/1299751110 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ce0101760 0x7f9ce019f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:03.307 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.325+0000 7f9ce71ff700 1 --2- 172.21.15.17:0/1299751110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ce0100dd0 0x7f9ce0198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:03.307 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.325+0000 7f9ce71ff700 1 -- 172.21.15.17:0/1299751110 >> 172.21.15.17:0/1299751110 conn(0x7f9ce00fb200 msgr2=0x7f9ce00fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:03.307 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.325+0000 7f9ce71ff700 1 -- 172.21.15.17:0/1299751110 shutdown_connections 2023-07-14T12:00:03.307 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.325+0000 7f9ce71ff700 1 -- 172.21.15.17:0/1299751110 wait complete. 2023-07-14T12:00:03.324 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T12:00:03.531 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.551+0000 7f7817dff700 1 -- 172.21.15.17:0/273586320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7810100dd0 msgr2=0x7f7810101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:03.532 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.551+0000 7f7817dff700 1 --2- 172.21.15.17:0/273586320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7810100dd0 0x7f7810101220 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f78000098f0 tx=0x7f7800009c00 comp rx=0 tx=0).stop 2023-07-14T12:00:03.532 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.551+0000 7f7817dff700 1 -- 172.21.15.17:0/273586320 shutdown_connections 2023-07-14T12:00:03.532 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.551+0000 7f7817dff700 1 --2- 172.21.15.17:0/273586320 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7810101760 0x7f7810108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:03.532 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.551+0000 7f7817dff700 1 --2- 172.21.15.17:0/273586320 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78100ffbe0 0x7f78100ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:03.532 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.551+0000 7f7817dff700 1 --2- 172.21.15.17:0/273586320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7810100dd0 0x7f7810101220 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:03.533 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.551+0000 7f7817dff700 1 -- 172.21.15.17:0/273586320 >> 172.21.15.17:0/273586320 conn(0x7f78100fb200 msgr2=0x7f78100fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:03.533 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.551+0000 7f7817dff700 1 -- 172.21.15.17:0/273586320 shutdown_connections 2023-07-14T12:00:03.533 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.551+0000 7f7817dff700 1 -- 172.21.15.17:0/273586320 wait complete. 2023-07-14T12:00:03.534 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.552+0000 7f7817dff700 1 Processor -- start 2023-07-14T12:00:03.535 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.552+0000 7f7817dff700 1 -- start start 2023-07-14T12:00:03.535 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.552+0000 7f7817dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78100ffbe0 0x7f7810197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:03.535 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.552+0000 7f7817dff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7810100dd0 0x7f7810197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:03.535 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.553+0000 7f7817dff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7810101760 0x7f781019f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:03.535 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.553+0000 7f7817dff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f781019f630 con 0x7f7810101760 2023-07-14T12:00:03.535 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.553+0000 7f7817dff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f781019f770 con 0x7f7810100dd0 2023-07-14T12:00:03.536 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.553+0000 7f7817dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f781019f900 con 0x7f78100ffbe0 2023-07-14T12:00:03.536 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.553+0000 7f7815b9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78100ffbe0 0x7f7810197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:03.536 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.553+0000 7f781639c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7810101760 0x7f781019f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:03.536 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.553+0000 7f7815b9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78100ffbe0 0x7f7810197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.17:52718/0 (socket says 172.21.15.17:52718) 2023-07-14T12:00:03.536 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.553+0000 7f7815b9b700 1 -- 172.21.15.17:0/4113934948 learned_addr learned my addr 172.21.15.17:0/4113934948 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:00:03.536 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.553+0000 7f781539a700 1 --2- 172.21.15.17:0/4113934948 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7810100dd0 0x7f7810197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:03.536 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.553+0000 7f7815b9b700 1 -- 172.21.15.17:0/4113934948 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7810100dd0 msgr2=0x7f7810197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:03.537 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.553+0000 7f7815b9b700 1 --2- 172.21.15.17:0/4113934948 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7810100dd0 0x7f7810197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:03.537 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.553+0000 7f7815b9b700 1 -- 172.21.15.17:0/4113934948 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7810101760 msgr2=0x7f781019f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:03.537 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.553+0000 7f7815b9b700 1 --2- 172.21.15.17:0/4113934948 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7810101760 0x7f781019f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:03.537 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.553+0000 7f7815b9b700 1 -- 172.21.15.17:0/4113934948 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7800009520 con 0x7f78100ffbe0 2023-07-14T12:00:03.537 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.553+0000 7f781639c700 1 --2- 172.21.15.17:0/4113934948 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7810101760 0x7f781019f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T12:00:03.537 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.553+0000 7f7815b9b700 1 --2- 172.21.15.17:0/4113934948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78100ffbe0 0x7f7810197ab0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f780c00e9f0 tx=0x7f780c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:03.537 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.554+0000 7f7806ffd700 1 -- 172.21.15.17:0/4113934948 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780c004220 con 0x7f78100ffbe0 2023-07-14T12:00:03.538 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.554+0000 7f7806ffd700 1 -- 172.21.15.17:0/4113934948 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f780c004380 con 0x7f78100ffbe0 2023-07-14T12:00:03.538 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.554+0000 7f7806ffd700 1 -- 172.21.15.17:0/4113934948 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780c003be0 con 0x7f78100ffbe0 2023-07-14T12:00:03.538 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.554+0000 7f7817dff700 1 -- 172.21.15.17:0/4113934948 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f781019fbe0 con 0x7f78100ffbe0 2023-07-14T12:00:03.538 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.554+0000 7f7817dff700 1 -- 172.21.15.17:0/4113934948 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78101a00a0 con 0x7f78100ffbe0 2023-07-14T12:00:03.539 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.555+0000 7f7817dff700 1 -- 172.21.15.17:0/4113934948 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f781010b350 con 0x7f78100ffbe0 2023-07-14T12:00:03.539 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.557+0000 7f7806ffd700 1 -- 172.21.15.17:0/4113934948 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f780c003d40 con 0x7f78100ffbe0 2023-07-14T12:00:03.539 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.559+0000 7f7806ffd700 1 --2- 172.21.15.17:0/4113934948 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f77fc0a1150 0x7f77fc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:03.539 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.559+0000 7f7806ffd700 1 -- 172.21.15.17:0/4113934948 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f780c014070 con 0x7f78100ffbe0 2023-07-14T12:00:03.539 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.559+0000 7f781539a700 1 --2- 172.21.15.17:0/4113934948 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f77fc0a1150 0x7f77fc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:03.540 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.559+0000 7f781539a700 1 --2- 172.21.15.17:0/4113934948 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f77fc0a1150 0x7f77fc0a3600 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f780000fa40 tx=0x7f7800019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:03.542 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.563+0000 7f7806ffd700 1 -- 172.21.15.17:0/4113934948 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f780c096100 con 0x7f78100ffbe0 2023-07-14T12:00:03.751 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.771+0000 7f7817dff700 1 -- 172.21.15.17:0/4113934948 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f78101a07e0 con 0x7f78100ffbe0 2023-07-14T12:00:03.752 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.772+0000 7f7806ffd700 1 -- 172.21.15.17:0/4113934948 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f780c099720 con 0x7f78100ffbe0 2023-07-14T12:00:03.753 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2023-07-14T12:00:03.755 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.774+0000 7f7817dff700 1 -- 172.21.15.17:0/4113934948 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f77fc0a1150 msgr2=0x7f77fc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:03.755 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.774+0000 7f7817dff700 1 --2- 172.21.15.17:0/4113934948 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f77fc0a1150 0x7f77fc0a3600 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f780000fa40 tx=0x7f7800019040 comp rx=0 tx=0).stop 2023-07-14T12:00:03.755 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.774+0000 7f7817dff700 1 -- 172.21.15.17:0/4113934948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78100ffbe0 msgr2=0x7f7810197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:03.755 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.774+0000 7f7817dff700 1 --2- 172.21.15.17:0/4113934948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78100ffbe0 0x7f7810197ab0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f780c00e9f0 tx=0x7f780c00ed00 comp rx=0 tx=0).stop 2023-07-14T12:00:03.756 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.774+0000 7f7817dff700 1 -- 172.21.15.17:0/4113934948 shutdown_connections 2023-07-14T12:00:03.756 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.774+0000 7f7817dff700 1 --2- 172.21.15.17:0/4113934948 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f77fc0a1150 0x7f77fc0a3600 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:03.756 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.774+0000 7f7817dff700 1 --2- 172.21.15.17:0/4113934948 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7810101760 0x7f781019f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:03.756 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.774+0000 7f7817dff700 1 --2- 172.21.15.17:0/4113934948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78100ffbe0 0x7f7810197ab0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:03.756 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.774+0000 7f7817dff700 1 --2- 172.21.15.17:0/4113934948 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7810100dd0 0x7f7810197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:03.756 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.774+0000 7f7817dff700 1 -- 172.21.15.17:0/4113934948 >> 172.21.15.17:0/4113934948 conn(0x7f78100fb200 msgr2=0x7f78100fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:03.756 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.774+0000 7f7817dff700 1 -- 172.21.15.17:0/4113934948 shutdown_connections 2023-07-14T12:00:03.756 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:03.775+0000 7f7817dff700 1 -- 172.21.15.17:0/4113934948 wait complete. 2023-07-14T12:00:03.776 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T12:00:03.983 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.002+0000 7f012adff700 1 -- 172.21.15.17:0/1635295901 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0124101760 msgr2=0x7f0124108020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:03.983 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.002+0000 7f012adff700 1 --2- 172.21.15.17:0/1635295901 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0124101760 0x7f0124108020 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f011c00b0a0 tx=0x7f011c00b3b0 comp rx=0 tx=0).stop 2023-07-14T12:00:03.984 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.003+0000 7f012adff700 1 -- 172.21.15.17:0/1635295901 shutdown_connections 2023-07-14T12:00:03.984 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.003+0000 7f012adff700 1 --2- 172.21.15.17:0/1635295901 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0124101760 0x7f0124108020 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:03.984 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.003+0000 7f012adff700 1 --2- 172.21.15.17:0/1635295901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01240ffbe0 0x7f01240ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:03.984 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.003+0000 7f012adff700 1 --2- 172.21.15.17:0/1635295901 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0124100dd0 0x7f0124101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:03.984 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.003+0000 7f012adff700 1 -- 172.21.15.17:0/1635295901 >> 172.21.15.17:0/1635295901 conn(0x7f01240fb200 msgr2=0x7f01240fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:03.984 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.003+0000 7f012adff700 1 -- 172.21.15.17:0/1635295901 shutdown_connections 2023-07-14T12:00:03.984 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.003+0000 7f012adff700 1 -- 172.21.15.17:0/1635295901 wait complete. 2023-07-14T12:00:03.986 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.004+0000 7f012adff700 1 Processor -- start 2023-07-14T12:00:03.986 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.004+0000 7f012adff700 1 -- start start 2023-07-14T12:00:03.986 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.004+0000 7f012adff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f01240ffbe0 0x7f0124195a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:03.986 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.004+0000 7f012adff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0124100dd0 0x7f0124195f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:03.987 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.004+0000 7f012adff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0124101760 0x7f012419cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:03.987 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.004+0000 7f012adff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012419d5a0 con 0x7f0124100dd0 2023-07-14T12:00:03.987 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.004+0000 7f012adff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012419d6e0 con 0x7f01240ffbe0 2023-07-14T12:00:03.987 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.004+0000 7f012adff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012419d870 con 0x7f0124101760 2023-07-14T12:00:03.987 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.004+0000 7f011bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0124100dd0 0x7f0124195f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:03.987 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.004+0000 7f011bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0124100dd0 0x7f0124195f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38174/0 (socket says 172.21.15.17:38174) 2023-07-14T12:00:03.987 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.004+0000 7f011bfff700 1 -- 172.21.15.17:0/1820343766 learned_addr learned my addr 172.21.15.17:0/1820343766 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:00:03.988 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.005+0000 7f012939c700 1 --2- 172.21.15.17:0/1820343766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0124101760 0x7f012419cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:03.988 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.005+0000 7f0128b9b700 1 --2- 172.21.15.17:0/1820343766 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f01240ffbe0 0x7f0124195a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:03.988 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.005+0000 7f011bfff700 1 -- 172.21.15.17:0/1820343766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0124101760 msgr2=0x7f012419cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:03.988 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.005+0000 7f011bfff700 1 --2- 172.21.15.17:0/1820343766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0124101760 0x7f012419cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:03.988 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.005+0000 7f011bfff700 1 -- 172.21.15.17:0/1820343766 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f01240ffbe0 msgr2=0x7f0124195a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:03.988 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.005+0000 7f011bfff700 1 --2- 172.21.15.17:0/1820343766 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f01240ffbe0 0x7f0124195a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:03.989 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.005+0000 7f011bfff700 1 -- 172.21.15.17:0/1820343766 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f011c009d00 con 0x7f0124100dd0 2023-07-14T12:00:03.989 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.005+0000 7f011bfff700 1 --2- 172.21.15.17:0/1820343766 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0124100dd0 0x7f0124195f60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f011400e9a0 tx=0x7f011400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:03.989 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.005+0000 7f0119ffb700 1 -- 172.21.15.17:0/1820343766 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0114009790 con 0x7f0124100dd0 2023-07-14T12:00:03.989 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.005+0000 7f0119ffb700 1 -- 172.21.15.17:0/1820343766 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0114010430 con 0x7f0124100dd0 2023-07-14T12:00:03.990 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.005+0000 7f012adff700 1 -- 172.21.15.17:0/1820343766 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f012419db50 con 0x7f0124100dd0 2023-07-14T12:00:03.990 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.005+0000 7f0119ffb700 1 -- 172.21.15.17:0/1820343766 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0114018680 con 0x7f0124100dd0 2023-07-14T12:00:03.990 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.006+0000 7f012adff700 1 -- 172.21.15.17:0/1820343766 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f012419e0b0 con 0x7f0124100dd0 2023-07-14T12:00:03.991 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.007+0000 7f012adff700 1 -- 172.21.15.17:0/1820343766 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f012410b410 con 0x7f0124100dd0 2023-07-14T12:00:03.991 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.009+0000 7f0119ffb700 1 -- 172.21.15.17:0/1820343766 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f01140105a0 con 0x7f0124100dd0 2023-07-14T12:00:03.991 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.011+0000 7f0119ffb700 1 --2- 172.21.15.17:0/1820343766 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f010c0a11d0 0x7f010c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:03.991 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.011+0000 7f0119ffb700 1 -- 172.21.15.17:0/1820343766 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f0114014070 con 0x7f0124100dd0 2023-07-14T12:00:03.991 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.011+0000 7f0128b9b700 1 --2- 172.21.15.17:0/1820343766 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f010c0a11d0 0x7f010c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:03.992 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.011+0000 7f0128b9b700 1 --2- 172.21.15.17:0/1820343766 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f010c0a11d0 0x7f010c0a3680 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0124100c30 tx=0x7f0120009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:03.995 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.015+0000 7f0119ffb700 1 -- 172.21.15.17:0/1820343766 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0114051790 con 0x7f0124100dd0 2023-07-14T12:00:04.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:03 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/2419512479' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T12:00:04.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:03 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/1299751110' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T12:00:04.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:03 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/4113934948' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T12:00:04.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:03 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/2419512479' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T12:00:04.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:03 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/1299751110' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T12:00:04.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:03 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/4113934948' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T12:00:04.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:03 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/2419512479' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T12:00:04.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:03 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/1299751110' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T12:00:04.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:03 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/4113934948' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T12:00:04.201 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.220+0000 7f012adff700 1 -- 172.21.15.17:0/1820343766 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f01241069f0 con 0x7f0124100dd0 2023-07-14T12:00:04.202 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.220+0000 7f0119ffb700 1 -- 172.21.15.17:0/1820343766 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f01140961b0 con 0x7f0124100dd0 2023-07-14T12:00:04.202 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2023-07-14T12:00:04.204 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.223+0000 7f012adff700 1 -- 172.21.15.17:0/1820343766 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f010c0a11d0 msgr2=0x7f010c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:04.204 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.223+0000 7f012adff700 1 --2- 172.21.15.17:0/1820343766 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f010c0a11d0 0x7f010c0a3680 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0124100c30 tx=0x7f0120009260 comp rx=0 tx=0).stop 2023-07-14T12:00:04.204 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.223+0000 7f012adff700 1 -- 172.21.15.17:0/1820343766 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0124100dd0 msgr2=0x7f0124195f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:04.205 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.223+0000 7f012adff700 1 --2- 172.21.15.17:0/1820343766 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0124100dd0 0x7f0124195f60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f011400e9a0 tx=0x7f011400ecb0 comp rx=0 tx=0).stop 2023-07-14T12:00:04.205 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.223+0000 7f012adff700 1 -- 172.21.15.17:0/1820343766 shutdown_connections 2023-07-14T12:00:04.205 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.223+0000 7f012adff700 1 --2- 172.21.15.17:0/1820343766 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f010c0a11d0 0x7f010c0a3680 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:04.205 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.223+0000 7f012adff700 1 --2- 172.21.15.17:0/1820343766 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f01240ffbe0 0x7f0124195a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:04.205 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.223+0000 7f012adff700 1 --2- 172.21.15.17:0/1820343766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0124101760 0x7f012419cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:04.205 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.223+0000 7f012adff700 1 --2- 172.21.15.17:0/1820343766 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0124100dd0 0x7f0124195f60 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:04.205 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.223+0000 7f012adff700 1 -- 172.21.15.17:0/1820343766 >> 172.21.15.17:0/1820343766 conn(0x7f01240fb200 msgr2=0x7f01240fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:04.206 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.223+0000 7f012adff700 1 -- 172.21.15.17:0/1820343766 shutdown_connections 2023-07-14T12:00:04.206 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.223+0000 7f012adff700 1 -- 172.21.15.17:0/1820343766 wait complete. 2023-07-14T12:00:04.222 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-14T12:00:04.428 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.447+0000 7f21563ff700 1 -- 172.21.15.17:0/3298607040 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2150100dd0 msgr2=0x7f2150101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:04.428 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.447+0000 7f21563ff700 1 --2- 172.21.15.17:0/3298607040 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2150100dd0 0x7f2150101220 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2144009990 tx=0x7f2144009ca0 comp rx=0 tx=0).stop 2023-07-14T12:00:04.429 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.447+0000 7f21563ff700 1 -- 172.21.15.17:0/3298607040 shutdown_connections 2023-07-14T12:00:04.429 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.447+0000 7f21563ff700 1 --2- 172.21.15.17:0/3298607040 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2150101760 0x7f2150108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:04.429 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.447+0000 7f21563ff700 1 --2- 172.21.15.17:0/3298607040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21500ffbe0 0x7f21500ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:04.429 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.447+0000 7f21563ff700 1 --2- 172.21.15.17:0/3298607040 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2150100dd0 0x7f2150101220 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:04.429 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.447+0000 7f21563ff700 1 -- 172.21.15.17:0/3298607040 >> 172.21.15.17:0/3298607040 conn(0x7f21500fb200 msgr2=0x7f21500fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:04.430 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.447+0000 7f21563ff700 1 -- 172.21.15.17:0/3298607040 shutdown_connections 2023-07-14T12:00:04.430 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.447+0000 7f21563ff700 1 -- 172.21.15.17:0/3298607040 wait complete. 2023-07-14T12:00:04.431 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.448+0000 7f21563ff700 1 Processor -- start 2023-07-14T12:00:04.432 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.449+0000 7f21563ff700 1 -- start start 2023-07-14T12:00:04.432 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.449+0000 7f21563ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f21500ffbe0 0x7f21500742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:04.432 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.449+0000 7f21563ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2150100dd0 0x7f2150072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:04.432 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.449+0000 7f21563ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2150101760 0x7f2150072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:04.432 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.449+0000 7f21563ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21500735e0 con 0x7f2150100dd0 2023-07-14T12:00:04.432 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.449+0000 7f21563ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2150073750 con 0x7f21500ffbe0 2023-07-14T12:00:04.433 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.449+0000 7f21563ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f215019f970 con 0x7f2150101760 2023-07-14T12:00:04.433 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.449+0000 7f214f7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2150100dd0 0x7f2150072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:04.433 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.449+0000 7f214f7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2150100dd0 0x7f2150072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38184/0 (socket says 172.21.15.17:38184) 2023-07-14T12:00:04.433 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.449+0000 7f214f7fe700 1 -- 172.21.15.17:0/4123007594 learned_addr learned my addr 172.21.15.17:0/4123007594 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:00:04.433 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.449+0000 7f214ffff700 1 --2- 172.21.15.17:0/4123007594 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f21500ffbe0 0x7f21500742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:04.433 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.449+0000 7f215499c700 1 --2- 172.21.15.17:0/4123007594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2150101760 0x7f2150072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:04.433 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.449+0000 7f214f7fe700 1 -- 172.21.15.17:0/4123007594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2150101760 msgr2=0x7f2150072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:04.434 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.449+0000 7f214f7fe700 1 --2- 172.21.15.17:0/4123007594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2150101760 0x7f2150072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:04.434 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.450+0000 7f214f7fe700 1 -- 172.21.15.17:0/4123007594 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f21500ffbe0 msgr2=0x7f21500742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:04.434 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.450+0000 7f214f7fe700 1 --2- 172.21.15.17:0/4123007594 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f21500ffbe0 0x7f21500742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:04.434 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.450+0000 7f214f7fe700 1 -- 172.21.15.17:0/4123007594 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21440095f0 con 0x7f2150100dd0 2023-07-14T12:00:04.434 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.450+0000 7f215499c700 1 --2- 172.21.15.17:0/4123007594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2150101760 0x7f2150072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T12:00:04.434 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.450+0000 7f214f7fe700 1 --2- 172.21.15.17:0/4123007594 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2150100dd0 0x7f2150072900 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2144004760 tx=0x7f2144004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:04.435 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.450+0000 7f214d7fa700 1 -- 172.21.15.17:0/4123007594 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214401d070 con 0x7f2150100dd0 2023-07-14T12:00:04.435 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.450+0000 7f214d7fa700 1 -- 172.21.15.17:0/4123007594 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2144004830 con 0x7f2150100dd0 2023-07-14T12:00:04.435 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.450+0000 7f214d7fa700 1 -- 172.21.15.17:0/4123007594 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2144005480 con 0x7f2150100dd0 2023-07-14T12:00:04.435 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.450+0000 7f21563ff700 1 -- 172.21.15.17:0/4123007594 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f215019fbf0 con 0x7f2150100dd0 2023-07-14T12:00:04.435 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.450+0000 7f21563ff700 1 -- 172.21.15.17:0/4123007594 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f215019ffb0 con 0x7f2150100dd0 2023-07-14T12:00:04.435 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.451+0000 7f21563ff700 1 -- 172.21.15.17:0/4123007594 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2150106ac0 con 0x7f2150100dd0 2023-07-14T12:00:04.436 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.453+0000 7f214d7fa700 1 -- 172.21.15.17:0/4123007594 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f214400fad0 con 0x7f2150100dd0 2023-07-14T12:00:04.436 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.454+0000 7f214d7fa700 1 --2- 172.21.15.17:0/4123007594 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f21380a11d0 0x7f21380a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:04.437 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.456+0000 7f214d7fa700 1 -- 172.21.15.17:0/4123007594 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f21440cde80 con 0x7f2150100dd0 2023-07-14T12:00:04.437 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.456+0000 7f214ffff700 1 --2- 172.21.15.17:0/4123007594 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f21380a11d0 0x7f21380a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:04.437 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.456+0000 7f214ffff700 1 --2- 172.21.15.17:0/4123007594 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f21380a11d0 0x7f21380a3680 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2150100c30 tx=0x7f2140009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:04.438 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.458+0000 7f214d7fa700 1 -- 172.21.15.17:0/4123007594 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2144097410 con 0x7f2150100dd0 2023-07-14T12:00:04.644 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.663+0000 7f21563ff700 1 -- 172.21.15.17:0/4123007594 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f215002c9e0 con 0x7f2150100dd0 2023-07-14T12:00:04.645 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.664+0000 7f214d7fa700 1 -- 172.21.15.17:0/4123007594 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7f214409aa30 con 0x7f2150100dd0 2023-07-14T12:00:04.645 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2023-07-14T12:00:04.647 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.666+0000 7f21563ff700 1 -- 172.21.15.17:0/4123007594 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f21380a11d0 msgr2=0x7f21380a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:04.647 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.666+0000 7f21563ff700 1 --2- 172.21.15.17:0/4123007594 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f21380a11d0 0x7f21380a3680 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2150100c30 tx=0x7f2140009260 comp rx=0 tx=0).stop 2023-07-14T12:00:04.647 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.666+0000 7f21563ff700 1 -- 172.21.15.17:0/4123007594 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2150100dd0 msgr2=0x7f2150072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:04.647 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.666+0000 7f21563ff700 1 --2- 172.21.15.17:0/4123007594 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2150100dd0 0x7f2150072900 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2144004760 tx=0x7f2144004790 comp rx=0 tx=0).stop 2023-07-14T12:00:04.648 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.666+0000 7f21563ff700 1 -- 172.21.15.17:0/4123007594 shutdown_connections 2023-07-14T12:00:04.648 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.666+0000 7f21563ff700 1 --2- 172.21.15.17:0/4123007594 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f21380a11d0 0x7f21380a3680 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:04.648 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.666+0000 7f21563ff700 1 --2- 172.21.15.17:0/4123007594 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f21500ffbe0 0x7f21500742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:04.648 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.666+0000 7f21563ff700 1 --2- 172.21.15.17:0/4123007594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2150101760 0x7f2150072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:04.648 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.666+0000 7f21563ff700 1 --2- 172.21.15.17:0/4123007594 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2150100dd0 0x7f2150072900 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:04.648 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.666+0000 7f21563ff700 1 -- 172.21.15.17:0/4123007594 >> 172.21.15.17:0/4123007594 conn(0x7f21500fb200 msgr2=0x7f21500fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:04.648 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.667+0000 7f21563ff700 1 -- 172.21.15.17:0/4123007594 shutdown_connections 2023-07-14T12:00:04.649 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.667+0000 7f21563ff700 1 -- 172.21.15.17:0/4123007594 wait complete. 2023-07-14T12:00:04.665 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-14T12:00:04.871 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.891+0000 7f433f7ff700 1 -- 172.21.15.17:0/3267390737 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4338100e50 msgr2=0x7f43381012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:04.871 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.891+0000 7f433f7ff700 1 --2- 172.21.15.17:0/3267390737 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4338100e50 0x7f43381012c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4328009990 tx=0x7f4328009ca0 comp rx=0 tx=0).stop 2023-07-14T12:00:04.871 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.891+0000 7f433f7ff700 1 -- 172.21.15.17:0/3267390737 shutdown_connections 2023-07-14T12:00:04.871 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.891+0000 7f433f7ff700 1 --2- 172.21.15.17:0/3267390737 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4338101800 0x7f43381080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:04.871 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.891+0000 7f433f7ff700 1 --2- 172.21.15.17:0/3267390737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43380ffbc0 0x7f43380fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:04.872 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.891+0000 7f433f7ff700 1 --2- 172.21.15.17:0/3267390737 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4338100e50 0x7f43381012c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:04.872 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.891+0000 7f433f7ff700 1 -- 172.21.15.17:0/3267390737 >> 172.21.15.17:0/3267390737 conn(0x7f43380fb200 msgr2=0x7f43380fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:04.872 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.891+0000 7f433f7ff700 1 -- 172.21.15.17:0/3267390737 shutdown_connections 2023-07-14T12:00:04.872 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.891+0000 7f433f7ff700 1 -- 172.21.15.17:0/3267390737 wait complete. 2023-07-14T12:00:04.873 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.892+0000 7f433f7ff700 1 Processor -- start 2023-07-14T12:00:04.874 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.892+0000 7f433f7ff700 1 -- start start 2023-07-14T12:00:04.874 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.893+0000 7f433f7ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f43380ffbc0 0x7f4338197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:04.874 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.893+0000 7f433f7ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4338100e50 0x7f4338198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:04.874 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.893+0000 7f433f7ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4338101800 0x7f433819f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:04.874 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.893+0000 7f433f7ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433819f690 con 0x7f4338101800 2023-07-14T12:00:04.874 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.893+0000 7f433f7ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433819f7d0 con 0x7f43380ffbc0 2023-07-14T12:00:04.874 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.893+0000 7f433f7ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433819f960 con 0x7f4338100e50 2023-07-14T12:00:04.874 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.893+0000 7f433dd9c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4338101800 0x7f433819f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:04.875 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.893+0000 7f433d59b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f43380ffbc0 0x7f4338197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:04.875 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.893+0000 7f433dd9c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4338101800 0x7f433819f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38198/0 (socket says 172.21.15.17:38198) 2023-07-14T12:00:04.875 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.893+0000 7f433dd9c700 1 -- 172.21.15.17:0/224185941 learned_addr learned my addr 172.21.15.17:0/224185941 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:00:04.875 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.893+0000 7f433cd9a700 1 --2- 172.21.15.17:0/224185941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4338100e50 0x7f4338198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:04.875 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.893+0000 7f433dd9c700 1 -- 172.21.15.17:0/224185941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4338100e50 msgr2=0x7f4338198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:04.875 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.893+0000 7f433dd9c700 1 --2- 172.21.15.17:0/224185941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4338100e50 0x7f4338198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:04.875 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.893+0000 7f433dd9c700 1 -- 172.21.15.17:0/224185941 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f43380ffbc0 msgr2=0x7f4338197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:04.876 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.893+0000 7f433dd9c700 1 --2- 172.21.15.17:0/224185941 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f43380ffbc0 0x7f4338197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:04.876 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.893+0000 7f433dd9c700 1 -- 172.21.15.17:0/224185941 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43280095f0 con 0x7f4338101800 2023-07-14T12:00:04.876 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.894+0000 7f433dd9c700 1 --2- 172.21.15.17:0/224185941 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4338101800 0x7f433819f0c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4330007ee0 tx=0x7f433000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:04.876 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.894+0000 7f432e7fc700 1 -- 172.21.15.17:0/224185941 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433000dd20 con 0x7f4338101800 2023-07-14T12:00:04.876 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.894+0000 7f432e7fc700 1 -- 172.21.15.17:0/224185941 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f433000f040 con 0x7f4338101800 2023-07-14T12:00:04.876 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.894+0000 7f433f7ff700 1 -- 172.21.15.17:0/224185941 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f433819fc40 con 0x7f4338101800 2023-07-14T12:00:04.876 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.894+0000 7f432e7fc700 1 -- 172.21.15.17:0/224185941 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43300048a0 con 0x7f4338101800 2023-07-14T12:00:04.876 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.894+0000 7f433f7ff700 1 -- 172.21.15.17:0/224185941 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43381a01a0 con 0x7f4338101800 2023-07-14T12:00:04.877 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.895+0000 7f433f7ff700 1 -- 172.21.15.17:0/224185941 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f433810b3e0 con 0x7f4338101800 2023-07-14T12:00:04.877 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.897+0000 7f432e7fc700 1 -- 172.21.15.17:0/224185941 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4330003a00 con 0x7f4338101800 2023-07-14T12:00:04.878 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.898+0000 7f432e7fc700 1 --2- 172.21.15.17:0/224185941 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f43240a11d0 0x7f43240a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:04.878 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.899+0000 7f432e7fc700 1 -- 172.21.15.17:0/224185941 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f43300cc260 con 0x7f4338101800 2023-07-14T12:00:04.878 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.899+0000 7f433d59b700 1 --2- 172.21.15.17:0/224185941 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f43240a11d0 0x7f43240a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:04.879 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.899+0000 7f433d59b700 1 --2- 172.21.15.17:0/224185941 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f43240a11d0 0x7f43240a3680 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4334009c60 tx=0x7f4334009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:04.882 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:04.902+0000 7f432e7fc700 1 -- 172.21.15.17:0/224185941 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f433004e790 con 0x7f4338101800 2023-07-14T12:00:05.109 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:05.128+0000 7f433f7ff700 1 -- 172.21.15.17:0/224185941 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f433810b5a0 con 0x7f4338101800 2023-07-14T12:00:05.109 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:05.129+0000 7f432e7fc700 1 -- 172.21.15.17:0/224185941 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) v1 ==== 81+0+592 (secure 0 0 0) 0x7f43300957c0 con 0x7f4338101800 2023-07-14T12:00:05.109 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/2084033276 2023-07-15T11:53:05.639597+0000 2023-07-14T12:00:05.109 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:6801/1059284716 2023-07-15T11:53:05.639597+0000 2023-07-14T12:00:05.110 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/3651785636 2023-07-15T11:53:05.639597+0000 2023-07-14T12:00:05.110 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/1373538024 2023-07-15T11:52:37.887946+0000 2023-07-14T12:00:05.110 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/3560395020 2023-07-15T11:52:37.887946+0000 2023-07-14T12:00:05.110 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:6800/1059284716 2023-07-15T11:53:05.639597+0000 2023-07-14T12:00:05.110 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/4262912861 2023-07-15T11:53:05.639597+0000 2023-07-14T12:00:05.110 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:0/2258375202 2023-07-15T11:52:37.887946+0000 2023-07-14T12:00:05.110 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:6800/2804227742 2023-07-15T11:52:37.887946+0000 2023-07-14T12:00:05.111 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17:6801/2804227742 2023-07-15T11:52:37.887946+0000 2023-07-14T12:00:05.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:04 smithi093 ceph-mon[128922]: pgmap v287: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-07-14T12:00:05.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:04 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/1820343766' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T12:00:05.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:04 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/4123007594' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T12:00:05.112 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:05.132+0000 7f433f7ff700 1 -- 172.21.15.17:0/224185941 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f43240a11d0 msgr2=0x7f43240a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:05.112 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:05.132+0000 7f433f7ff700 1 --2- 172.21.15.17:0/224185941 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f43240a11d0 0x7f43240a3680 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4334009c60 tx=0x7f4334009260 comp rx=0 tx=0).stop 2023-07-14T12:00:05.113 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:05.132+0000 7f433f7ff700 1 -- 172.21.15.17:0/224185941 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4338101800 msgr2=0x7f433819f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:05.113 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:05.132+0000 7f433f7ff700 1 --2- 172.21.15.17:0/224185941 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4338101800 0x7f433819f0c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4330007ee0 tx=0x7f433000d3b0 comp rx=0 tx=0).stop 2023-07-14T12:00:05.113 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:05.132+0000 7f433f7ff700 1 -- 172.21.15.17:0/224185941 shutdown_connections 2023-07-14T12:00:05.113 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:05.132+0000 7f433f7ff700 1 --2- 172.21.15.17:0/224185941 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f43240a11d0 0x7f43240a3680 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:05.113 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:05.132+0000 7f433f7ff700 1 --2- 172.21.15.17:0/224185941 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f43380ffbc0 0x7f4338197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:05.113 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:05.132+0000 7f433f7ff700 1 --2- 172.21.15.17:0/224185941 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4338101800 0x7f433819f0c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:05.113 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:05.132+0000 7f433f7ff700 1 --2- 172.21.15.17:0/224185941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4338100e50 0x7f4338198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:05.114 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:05.132+0000 7f433f7ff700 1 -- 172.21.15.17:0/224185941 >> 172.21.15.17:0/224185941 conn(0x7f43380fb200 msgr2=0x7f43381066d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:05.114 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:05.132+0000 7f433f7ff700 1 -- 172.21.15.17:0/224185941 shutdown_connections 2023-07-14T12:00:05.114 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:05.132+0000 7f433f7ff700 1 -- 172.21.15.17:0/224185941 wait complete. 2023-07-14T12:00:05.114 INFO:teuthology.orchestra.run.smithi017.stderr:listed 10 entries 2023-07-14T12:00:05.127 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:04 smithi017 ceph-mon[126339]: pgmap v287: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-07-14T12:00:05.127 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:04 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/1820343766' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T12:00:05.128 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:04 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/4123007594' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T12:00:05.129 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi017.front.sepia.ceph.com... 2023-07-14T12:00:05.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T12:00:05.130 DEBUG:teuthology.orchestra.run.smithi017:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-07-14T12:00:05.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:04 smithi097 ceph-mon[129313]: pgmap v287: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-07-14T12:00:05.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:04 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/1820343766' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T12:00:05.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:04 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/4123007594' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T12:00:05.199 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi017.stderr:ceph-fuse[163668]: fuse finished with error 0 and tester_r 0 2023-07-14T12:00:05.203 INFO:teuthology.orchestra.run:waiting for 300 2023-07-14T12:00:06.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:05 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/224185941' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-14T12:00:06.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:05 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2023-07-14T12:00:06.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:05 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]: dispatch 2023-07-14T12:00:06.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:05 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2023-07-14T12:00:06.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:05 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2023-07-14T12:00:06.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:05 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2023-07-14T12:00:06.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:05 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2023-07-14T12:00:06.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:05 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2023-07-14T12:00:06.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:05 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 10]}]: dispatch 2023-07-14T12:00:06.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:05 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2023-07-14T12:00:06.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:05 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2023-07-14T12:00:06.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:05 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/224185941' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-14T12:00:06.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:05 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2023-07-14T12:00:06.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:05 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]: dispatch 2023-07-14T12:00:06.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:05 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2023-07-14T12:00:06.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:05 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2023-07-14T12:00:06.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:05 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2023-07-14T12:00:06.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:05 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2023-07-14T12:00:06.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:05 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2023-07-14T12:00:06.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:05 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 10]}]: dispatch 2023-07-14T12:00:06.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:05 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2023-07-14T12:00:06.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:05 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2023-07-14T12:00:06.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:05 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/224185941' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-14T12:00:06.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:05 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2023-07-14T12:00:06.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:05 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]: dispatch 2023-07-14T12:00:06.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:05 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2023-07-14T12:00:06.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:05 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2023-07-14T12:00:06.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:05 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2023-07-14T12:00:06.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:05 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2023-07-14T12:00:06.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:05 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2023-07-14T12:00:06.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:05 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 10]}]: dispatch 2023-07-14T12:00:06.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:05 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2023-07-14T12:00:06.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:05 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2023-07-14T12:00:07.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:06 smithi093 ceph-mon[128922]: pgmap v288: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-07-14T12:00:07.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:06 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2023-07-14T12:00:07.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:06 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2023-07-14T12:00:07.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:06 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2023-07-14T12:00:07.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:06 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2023-07-14T12:00:07.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:06 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2023-07-14T12:00:07.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:06 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2023-07-14T12:00:07.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:06 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2023-07-14T12:00:07.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:06 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 10]}]': finished 2023-07-14T12:00:07.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:06 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2023-07-14T12:00:07.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:06 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2023-07-14T12:00:07.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:06 smithi093 ceph-mon[128922]: osdmap e89: 12 total, 12 up, 12 in 2023-07-14T12:00:07.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:06 smithi097 ceph-mon[129313]: pgmap v288: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-07-14T12:00:07.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:06 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2023-07-14T12:00:07.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:06 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2023-07-14T12:00:07.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:06 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2023-07-14T12:00:07.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:06 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2023-07-14T12:00:07.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:06 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2023-07-14T12:00:07.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:06 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2023-07-14T12:00:07.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:06 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2023-07-14T12:00:07.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:06 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 10]}]': finished 2023-07-14T12:00:07.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:06 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2023-07-14T12:00:07.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:06 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2023-07-14T12:00:07.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:06 smithi097 ceph-mon[129313]: osdmap e89: 12 total, 12 up, 12 in 2023-07-14T12:00:07.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:06 smithi017 ceph-mon[126339]: pgmap v288: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-07-14T12:00:07.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:06 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2023-07-14T12:00:07.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:06 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2023-07-14T12:00:07.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:06 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2023-07-14T12:00:07.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:06 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2023-07-14T12:00:07.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:06 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2023-07-14T12:00:07.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:06 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2023-07-14T12:00:07.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:06 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2023-07-14T12:00:07.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:06 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 10]}]': finished 2023-07-14T12:00:07.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:06 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2023-07-14T12:00:07.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:06 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2023-07-14T12:00:07.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:06 smithi017 ceph-mon[126339]: osdmap e89: 12 total, 12 up, 12 in 2023-07-14T12:00:08.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:07 smithi093 ceph-mon[128922]: osdmap e90: 12 total, 12 up, 12 in 2023-07-14T12:00:08.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:07 smithi097 ceph-mon[129313]: osdmap e90: 12 total, 12 up, 12 in 2023-07-14T12:00:08.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:07 smithi017 ceph-mon[126339]: osdmap e90: 12 total, 12 up, 12 in 2023-07-14T12:00:09.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:08 smithi093 ceph-mon[128922]: pgmap v291: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s 2023-07-14T12:00:09.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:08 smithi097 ceph-mon[129313]: pgmap v291: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s 2023-07-14T12:00:09.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:08 smithi017 ceph-mon[126339]: pgmap v291: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s 2023-07-14T12:00:11.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:10 smithi093 ceph-mon[128922]: pgmap v292: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2023-07-14T12:00:11.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:10 smithi097 ceph-mon[129313]: pgmap v292: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2023-07-14T12:00:11.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:10 smithi017 ceph-mon[126339]: pgmap v292: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2023-07-14T12:00:11.305 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi017.front.sepia.ceph.com 2023-07-14T12:00:11.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T12:00:11.306 DEBUG:teuthology.orchestra.run.smithi017:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-07-14T12:00:11.368 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T12:00:11.577 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.597+0000 7feead1ff700 1 -- 172.21.15.17:0/2221669259 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feea8072ee0 msgr2=0x7feea8073350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:11.578 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.597+0000 7feead1ff700 1 --2- 172.21.15.17:0/2221669259 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feea8072ee0 0x7feea8073350 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fee9c009990 tx=0x7fee9c009ca0 comp rx=0 tx=0).stop 2023-07-14T12:00:11.578 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.598+0000 7feead1ff700 1 -- 172.21.15.17:0/2221669259 shutdown_connections 2023-07-14T12:00:11.578 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.598+0000 7feead1ff700 1 --2- 172.21.15.17:0/2221669259 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feea80744b0 0x7feea80729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:11.578 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.598+0000 7feead1ff700 1 --2- 172.21.15.17:0/2221669259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feea8107a20 0x7feea8109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:11.578 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.598+0000 7feead1ff700 1 --2- 172.21.15.17:0/2221669259 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feea8072ee0 0x7feea8073350 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:11.578 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.598+0000 7feead1ff700 1 -- 172.21.15.17:0/2221669259 >> 172.21.15.17:0/2221669259 conn(0x7feea80fb1c0 msgr2=0x7feea80fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:11.578 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.598+0000 7feead1ff700 1 -- 172.21.15.17:0/2221669259 shutdown_connections 2023-07-14T12:00:11.579 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.598+0000 7feead1ff700 1 -- 172.21.15.17:0/2221669259 wait complete. 2023-07-14T12:00:11.580 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.599+0000 7feead1ff700 1 Processor -- start 2023-07-14T12:00:11.580 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.599+0000 7feead1ff700 1 -- start start 2023-07-14T12:00:11.580 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.600+0000 7feead1ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feea8072ee0 0x7feea8197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:11.581 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.600+0000 7feead1ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feea80744b0 0x7feea8198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:11.581 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.600+0000 7feead1ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feea8107a20 0x7feea819f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:11.581 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.600+0000 7feead1ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea819f830 con 0x7feea8107a20 2023-07-14T12:00:11.581 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.600+0000 7feead1ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea819f9a0 con 0x7feea8072ee0 2023-07-14T12:00:11.581 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.600+0000 7feead1ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea819fb60 con 0x7feea80744b0 2023-07-14T12:00:11.581 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.600+0000 7feea759e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feea8107a20 0x7feea819f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:11.581 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.600+0000 7feea6d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feea8072ee0 0x7feea8197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:11.582 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.600+0000 7feea6d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feea8072ee0 0x7feea8197cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.17:51572/0 (socket says 172.21.15.17:51572) 2023-07-14T12:00:11.582 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.600+0000 7feea6d9d700 1 -- 172.21.15.17:0/1602635014 learned_addr learned my addr 172.21.15.17:0/1602635014 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:00:11.582 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.600+0000 7feea659c700 1 --2- 172.21.15.17:0/1602635014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feea80744b0 0x7feea8198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:11.582 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.601+0000 7feea759e700 1 -- 172.21.15.17:0/1602635014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feea80744b0 msgr2=0x7feea8198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:11.582 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.601+0000 7feea759e700 1 --2- 172.21.15.17:0/1602635014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feea80744b0 0x7feea8198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:11.582 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.601+0000 7feea759e700 1 -- 172.21.15.17:0/1602635014 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feea8072ee0 msgr2=0x7feea8197cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:11.583 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.601+0000 7feea759e700 1 --2- 172.21.15.17:0/1602635014 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feea8072ee0 0x7feea8197cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:11.583 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.601+0000 7feea759e700 1 -- 172.21.15.17:0/1602635014 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee9c0095f0 con 0x7feea8107a20 2023-07-14T12:00:11.583 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.601+0000 7feea6d9d700 1 --2- 172.21.15.17:0/1602635014 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feea8072ee0 0x7feea8197cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T12:00:11.583 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.601+0000 7feea759e700 1 --2- 172.21.15.17:0/1602635014 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feea8107a20 0x7feea819f260 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7feea0007f10 tx=0x7feea000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:11.584 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.601+0000 7fee8ffff700 1 -- 172.21.15.17:0/1602635014 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea0009ea0 con 0x7feea8107a20 2023-07-14T12:00:11.584 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.601+0000 7fee8ffff700 1 -- 172.21.15.17:0/1602635014 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feea000f040 con 0x7feea8107a20 2023-07-14T12:00:11.584 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.601+0000 7feead1ff700 1 -- 172.21.15.17:0/1602635014 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feea819fe40 con 0x7feea8107a20 2023-07-14T12:00:11.584 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.601+0000 7fee8ffff700 1 -- 172.21.15.17:0/1602635014 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea0004350 con 0x7feea8107a20 2023-07-14T12:00:11.585 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.601+0000 7feead1ff700 1 -- 172.21.15.17:0/1602635014 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feea81a0270 con 0x7feea8107a20 2023-07-14T12:00:11.585 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.602+0000 7feead1ff700 1 -- 172.21.15.17:0/1602635014 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feea810d5f0 con 0x7feea8107a20 2023-07-14T12:00:11.585 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.604+0000 7fee8ffff700 1 -- 172.21.15.17:0/1602635014 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7feea000db20 con 0x7feea8107a20 2023-07-14T12:00:11.585 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.606+0000 7fee8ffff700 1 --2- 172.21.15.17:0/1602635014 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fee900a11d0 0x7fee900a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:11.586 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.606+0000 7fee8ffff700 1 -- 172.21.15.17:0/1602635014 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7feea00cc460 con 0x7feea8107a20 2023-07-14T12:00:11.586 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.606+0000 7feea6d9d700 1 --2- 172.21.15.17:0/1602635014 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fee900a11d0 0x7fee900a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:11.586 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.607+0000 7feea6d9d700 1 --2- 172.21.15.17:0/1602635014 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fee900a11d0 0x7fee900a3680 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7feea81991c0 tx=0x7fee9800b4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:11.589 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.610+0000 7fee8ffff700 1 -- 172.21.15.17:0/1602635014 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7feea0095840 con 0x7feea8107a20 2023-07-14T12:00:11.805 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.825+0000 7feead1ff700 1 -- 172.21.15.17:0/1602635014 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feea81a0b10 con 0x7feea8107a20 2023-07-14T12:00:11.807 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.827+0000 7fee8ffff700 1 -- 172.21.15.17:0/1602635014 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24336 (secure 0 0 0) 0x7feea0098e60 con 0x7feea8107a20 2023-07-14T12:00:11.807 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T12:00:11.808 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":90,"fsid":"b3c386cc-223c-11ee-9b33-001a4aab830c","created":"2023-07-14T11:51:59.992960+0000","modified":"2023-07-14T12:00:06.800621+0000","last_up_change":"2023-07-14T11:58:55.037769+0000","last_in_change":"2023-07-14T11:58:37.114870+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:55:28.237002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:59:41.560502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T11:59:42.725461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:59:46.267029+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"57003438-a83c-4c8e-a029-56414740895c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6803","nonce":3588366384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6805","nonce":3588366384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6809","nonce":3588366384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6807","nonce":3588366384}]},"public_addr":"172.21.15.17:6803/3588366384","cluster_addr":"172.21.15.17:6805/3588366384","heartbeat_back_addr":"172.21.15.17:6809/3588366384","heartbeat_front_addr":"172.21.15.17:6807/3588366384","state":["exists","up"]},{"osd":1,"uuid":"f56b6337-d8a9-4411-b9d8-b6863dc98556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6801","nonce":3147203064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6803","nonce":3147203064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6807","nonce":3147203064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6805","nonce":3147203064}]},"public_addr":"172.21.15.93:6801/3147203064","cluster_addr":"172.21.15.93:6803/3147203064","heartbeat_back_addr":"172.21.15.93:6807/3147203064","heartbeat_front_addr":"172.21.15.93:6805/3147203064","state":["exists","up"]},{"osd":2,"uuid":"fc34bb7f-551f-46e1-99c0-59924d6c1346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6801","nonce":1877269815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6803","nonce":1877269815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6807","nonce":1877269815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6805","nonce":1877269815}]},"public_addr":"172.21.15.97:6801/1877269815","cluster_addr":"172.21.15.97:6803/1877269815","heartbeat_back_addr":"172.21.15.97:6807/1877269815","heartbeat_front_addr":"172.21.15.97:6805/1877269815","state":["exists","up"]},{"osd":3,"uuid":"37a1583c-4fb8-412a-a0c8-f83b1ff26a91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6811","nonce":2784619821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6813","nonce":2784619821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6817","nonce":2784619821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6815","nonce":2784619821}]},"public_addr":"172.21.15.17:6811/2784619821","cluster_addr":"172.21.15.17:6813/2784619821","heartbeat_back_addr":"172.21.15.17:6817/2784619821","heartbeat_front_addr":"172.21.15.17:6815/2784619821","state":["exists","up"]},{"osd":4,"uuid":"f4b9f377-d519-4b14-ac4a-f1debaeed9f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6809","nonce":3151801173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6811","nonce":3151801173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6815","nonce":3151801173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6813","nonce":3151801173}]},"public_addr":"172.21.15.93:6809/3151801173","cluster_addr":"172.21.15.93:6811/3151801173","heartbeat_back_addr":"172.21.15.93:6815/3151801173","heartbeat_front_addr":"172.21.15.93:6813/3151801173","state":["exists","up"]},{"osd":5,"uuid":"2d2af354-0535-4ef9-b166-3ef219498db8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6809","nonce":2184413924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6811","nonce":2184413924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6815","nonce":2184413924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6813","nonce":2184413924}]},"public_addr":"172.21.15.97:6809/2184413924","cluster_addr":"172.21.15.97:6811/2184413924","heartbeat_back_addr":"172.21.15.97:6815/2184413924","heartbeat_front_addr":"172.21.15.97:6813/2184413924","state":["exists","up"]},{"osd":6,"uuid":"9cdf0c4b-62dc-4168-864e-a9d1650a9ea8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6819","nonce":50486581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6821","nonce":50486581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6825","nonce":50486581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6823","nonce":50486581}]},"public_addr":"172.21.15.17:6819/50486581","cluster_addr":"172.21.15.17:6821/50486581","heartbeat_back_addr":"172.21.15.17:6825/50486581","heartbeat_front_addr":"172.21.15.17:6823/50486581","state":["exists","up"]},{"osd":7,"uuid":"09f60823-5491-45cd-9324-6ae0d56ce262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6817","nonce":2596912392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6819","nonce":2596912392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6823","nonce":2596912392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6821","nonce":2596912392}]},"public_addr":"172.21.15.93:6817/2596912392","cluster_addr":"172.21.15.93:6819/2596912392","heartbeat_back_addr":"172.21.15.93:6823/2596912392","heartbeat_front_addr":"172.21.15.93:6821/2596912392","state":["exists","up"]},{"osd":8,"uuid":"8c5771c9-718e-4313-a2b6-5e1c37c53f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6817","nonce":1965492859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6819","nonce":1965492859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6823","nonce":1965492859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6821","nonce":1965492859}]},"public_addr":"172.21.15.97:6817/1965492859","cluster_addr":"172.21.15.97:6819/1965492859","heartbeat_back_addr":"172.21.15.97:6823/1965492859","heartbeat_front_addr":"172.21.15.97:6821/1965492859","state":["exists","up"]},{"osd":9,"uuid":"1197b8a3-c09f-4c57-aba1-9e988a1620b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6827","nonce":2512645700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6829","nonce":2512645700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6833","nonce":2512645700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6831","nonce":2512645700}]},"public_addr":"172.21.15.17:6827/2512645700","cluster_addr":"172.21.15.17:6829/2512645700","heartbeat_back_addr":"172.21.15.17:6833/2512645700","heartbeat_front_addr":"172.21.15.17:6831/2512645700","state":["exists","up"]},{"osd":10,"uuid":"f99918c2-cdd8-4a7a-be2e-d1d6e0f828a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6825","nonce":264429558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6827","nonce":264429558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6831","nonce":264429558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6829","nonce":264429558}]},"public_addr":"172.21.15.93:6825/264429558","cluster_addr":"172.21.15.93:6827/264429558","heartbeat_back_addr":"172.21.15.93:6831/264429558","heartbeat_front_addr":"172.21.15.93:6829/264429558","state":["exists","up"]},{"osd":11,"uuid":"6bbb5b79-8d4d-441f-9583-203df6c78cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6825","nonce":3389913754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6827","nonce":3389913754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6831","nonce":3389913754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6829","nonce":3389913754}]},"public_addr":"172.21.15.97:6825/3389913754","cluster_addr":"172.21.15.97:6827/3389913754","heartbeat_back_addr":"172.21.15.97:6831/3389913754","heartbeat_front_addr":"172.21.15.97:6829/3389913754","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:54:41.079618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:03.207899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:24.136961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:50.392609+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:56:12.136237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:56:33.212314+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:00.243646+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:22.059234+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:43.232399+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:10.043896+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:31.910780+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:53.158983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.9","mappings":[{"from":5,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/2084033276":"2023-07-15T11:53:05.639597+0000","172.21.15.17:6801/1059284716":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/3651785636":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/1373538024":"2023-07-15T11:52:37.887946+0000","172.21.15.17:0/3560395020":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6800/1059284716":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/4262912861":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/2258375202":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6800/2804227742":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6801/2804227742":"2023-07-15T11:52:37.887946+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T12:00:11.809 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.830+0000 7feead1ff700 1 -- 172.21.15.17:0/1602635014 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fee900a11d0 msgr2=0x7fee900a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:11.809 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.830+0000 7feead1ff700 1 --2- 172.21.15.17:0/1602635014 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fee900a11d0 0x7fee900a3680 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7feea81991c0 tx=0x7fee9800b4a0 comp rx=0 tx=0).stop 2023-07-14T12:00:11.810 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.830+0000 7feead1ff700 1 -- 172.21.15.17:0/1602635014 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feea8107a20 msgr2=0x7feea819f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:11.810 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.830+0000 7feead1ff700 1 --2- 172.21.15.17:0/1602635014 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feea8107a20 0x7feea819f260 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7feea0007f10 tx=0x7feea000d3b0 comp rx=0 tx=0).stop 2023-07-14T12:00:11.810 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.830+0000 7feead1ff700 1 -- 172.21.15.17:0/1602635014 shutdown_connections 2023-07-14T12:00:11.810 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.830+0000 7feead1ff700 1 --2- 172.21.15.17:0/1602635014 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fee900a11d0 0x7fee900a3680 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:11.811 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.830+0000 7feead1ff700 1 --2- 172.21.15.17:0/1602635014 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feea8072ee0 0x7feea8197cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:11.811 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.830+0000 7feead1ff700 1 --2- 172.21.15.17:0/1602635014 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feea8107a20 0x7feea819f260 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:11.811 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.830+0000 7feead1ff700 1 --2- 172.21.15.17:0/1602635014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feea80744b0 0x7feea8198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:11.811 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.830+0000 7feead1ff700 1 -- 172.21.15.17:0/1602635014 >> 172.21.15.17:0/1602635014 conn(0x7feea80fb1c0 msgr2=0x7feea8109b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:11.811 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.830+0000 7feead1ff700 1 -- 172.21.15.17:0/1602635014 shutdown_connections 2023-07-14T12:00:11.811 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:11.830+0000 7feead1ff700 1 -- 172.21.15.17:0/1602635014 wait complete. 2023-07-14T12:00:11.829 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T12:00:12.036 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.056+0000 7f3c815ff700 1 -- 172.21.15.17:0/2470518394 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c7c102f40 msgr2=0x7f3c7c105320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:12.037 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.056+0000 7f3c815ff700 1 --2- 172.21.15.17:0/2470518394 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c7c102f40 0x7f3c7c105320 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3c64009990 tx=0x7f3c64009ca0 comp rx=0 tx=0).stop 2023-07-14T12:00:12.037 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.056+0000 7f3c815ff700 1 -- 172.21.15.17:0/2470518394 shutdown_connections 2023-07-14T12:00:12.037 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.056+0000 7f3c815ff700 1 --2- 172.21.15.17:0/2470518394 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c7c100620 0x7f3c7c102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:12.038 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.056+0000 7f3c815ff700 1 --2- 172.21.15.17:0/2470518394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c7c105860 0x7f3c7c107cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:12.038 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.056+0000 7f3c815ff700 1 --2- 172.21.15.17:0/2470518394 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c7c102f40 0x7f3c7c105320 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:12.038 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.056+0000 7f3c815ff700 1 -- 172.21.15.17:0/2470518394 >> 172.21.15.17:0/2470518394 conn(0x7f3c7c0fa120 msgr2=0x7f3c7c0fc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:12.039 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.056+0000 7f3c815ff700 1 -- 172.21.15.17:0/2470518394 shutdown_connections 2023-07-14T12:00:12.039 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.056+0000 7f3c815ff700 1 -- 172.21.15.17:0/2470518394 wait complete. 2023-07-14T12:00:12.041 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.057+0000 7f3c815ff700 1 Processor -- start 2023-07-14T12:00:12.041 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.057+0000 7f3c815ff700 1 -- start start 2023-07-14T12:00:12.041 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.058+0000 7f3c815ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c7c100620 0x7f3c7c197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:12.041 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.058+0000 7f3c815ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c7c102f40 0x7f3c7c197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:12.041 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.058+0000 7f3c815ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c7c105860 0x7f3c7c19f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:12.041 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.058+0000 7f3c815ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c19f630 con 0x7f3c7c102f40 2023-07-14T12:00:12.041 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.058+0000 7f3c815ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c19f770 con 0x7f3c7c105860 2023-07-14T12:00:12.042 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.058+0000 7f3c815ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c19f900 con 0x7f3c7c100620 2023-07-14T12:00:12.042 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.058+0000 7f3c7affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c7c100620 0x7f3c7c197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:12.042 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.058+0000 7f3c73fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c7c102f40 0x7f3c7c197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:12.042 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.058+0000 7f3c7affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c7c100620 0x7f3c7c197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.17:52808/0 (socket says 172.21.15.17:52808) 2023-07-14T12:00:12.042 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.058+0000 7f3c7affd700 1 -- 172.21.15.17:0/1713453134 learned_addr learned my addr 172.21.15.17:0/1713453134 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:00:12.042 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.058+0000 7f3c7b7fe700 1 --2- 172.21.15.17:0/1713453134 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c7c105860 0x7f3c7c19f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:12.042 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.059+0000 7f3c73fff700 1 -- 172.21.15.17:0/1713453134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c7c100620 msgr2=0x7f3c7c197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:12.043 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.059+0000 7f3c73fff700 1 --2- 172.21.15.17:0/1713453134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c7c100620 0x7f3c7c197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:12.043 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.059+0000 7f3c73fff700 1 -- 172.21.15.17:0/1713453134 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c7c105860 msgr2=0x7f3c7c19f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:12.043 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.059+0000 7f3c73fff700 1 --2- 172.21.15.17:0/1713453134 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c7c105860 0x7f3c7c19f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:12.043 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.059+0000 7f3c73fff700 1 -- 172.21.15.17:0/1713453134 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c640095f0 con 0x7f3c7c102f40 2023-07-14T12:00:12.043 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.059+0000 7f3c7affd700 1 --2- 172.21.15.17:0/1713453134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c7c100620 0x7f3c7c197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T12:00:12.043 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.059+0000 7f3c73fff700 1 --2- 172.21.15.17:0/1713453134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c7c102f40 0x7f3c7c197ff0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3c6400b680 tx=0x7f3c64009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:12.043 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.059+0000 7f3c78ff9700 1 -- 172.21.15.17:0/1713453134 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6401d070 con 0x7f3c7c102f40 2023-07-14T12:00:12.044 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.059+0000 7f3c78ff9700 1 -- 172.21.15.17:0/1713453134 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c64021410 con 0x7f3c7c102f40 2023-07-14T12:00:12.044 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.059+0000 7f3c815ff700 1 -- 172.21.15.17:0/1713453134 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c7c19fb80 con 0x7f3c7c102f40 2023-07-14T12:00:12.044 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.059+0000 7f3c78ff9700 1 -- 172.21.15.17:0/1713453134 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c64017650 con 0x7f3c7c102f40 2023-07-14T12:00:12.044 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.059+0000 7f3c815ff700 1 -- 172.21.15.17:0/1713453134 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c7c1a0000 con 0x7f3c7c102f40 2023-07-14T12:00:12.044 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.060+0000 7f3c815ff700 1 -- 172.21.15.17:0/1713453134 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c7c10b3b0 con 0x7f3c7c102f40 2023-07-14T12:00:12.045 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.062+0000 7f3c78ff9700 1 -- 172.21.15.17:0/1713453134 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3c64021810 con 0x7f3c7c102f40 2023-07-14T12:00:12.045 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.064+0000 7f3c78ff9700 1 --2- 172.21.15.17:0/1713453134 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3c680a11d0 0x7f3c680a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:12.045 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.064+0000 7f3c78ff9700 1 -- 172.21.15.17:0/1713453134 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3c640cdf70 con 0x7f3c7c102f40 2023-07-14T12:00:12.045 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.064+0000 7f3c7affd700 1 --2- 172.21.15.17:0/1713453134 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3c680a11d0 0x7f3c680a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:12.045 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.065+0000 7f3c7affd700 1 --2- 172.21.15.17:0/1713453134 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3c680a11d0 0x7f3c680a3680 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c011550 tx=0x7f3c6c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:12.047 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.067+0000 7f3c78ff9700 1 -- 172.21.15.17:0/1713453134 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3c640973e0 con 0x7f3c7c102f40 2023-07-14T12:00:12.257 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.278+0000 7f3c815ff700 1 -- 172.21.15.17:0/1713453134 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3c7c1a05b0 con 0x7f3c7c102f40 2023-07-14T12:00:12.259 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.279+0000 7f3c78ff9700 1 -- 172.21.15.17:0/1713453134 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24336 (secure 0 0 0) 0x7f3c6409aa00 con 0x7f3c7c102f40 2023-07-14T12:00:12.259 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T12:00:12.260 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":90,"fsid":"b3c386cc-223c-11ee-9b33-001a4aab830c","created":"2023-07-14T11:51:59.992960+0000","modified":"2023-07-14T12:00:06.800621+0000","last_up_change":"2023-07-14T11:58:55.037769+0000","last_in_change":"2023-07-14T11:58:37.114870+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:55:28.237002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:59:41.560502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T11:59:42.725461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:59:46.267029+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"57003438-a83c-4c8e-a029-56414740895c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6803","nonce":3588366384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6805","nonce":3588366384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6809","nonce":3588366384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6807","nonce":3588366384}]},"public_addr":"172.21.15.17:6803/3588366384","cluster_addr":"172.21.15.17:6805/3588366384","heartbeat_back_addr":"172.21.15.17:6809/3588366384","heartbeat_front_addr":"172.21.15.17:6807/3588366384","state":["exists","up"]},{"osd":1,"uuid":"f56b6337-d8a9-4411-b9d8-b6863dc98556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6801","nonce":3147203064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6803","nonce":3147203064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6807","nonce":3147203064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6805","nonce":3147203064}]},"public_addr":"172.21.15.93:6801/3147203064","cluster_addr":"172.21.15.93:6803/3147203064","heartbeat_back_addr":"172.21.15.93:6807/3147203064","heartbeat_front_addr":"172.21.15.93:6805/3147203064","state":["exists","up"]},{"osd":2,"uuid":"fc34bb7f-551f-46e1-99c0-59924d6c1346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6801","nonce":1877269815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6803","nonce":1877269815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6807","nonce":1877269815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6805","nonce":1877269815}]},"public_addr":"172.21.15.97:6801/1877269815","cluster_addr":"172.21.15.97:6803/1877269815","heartbeat_back_addr":"172.21.15.97:6807/1877269815","heartbeat_front_addr":"172.21.15.97:6805/1877269815","state":["exists","up"]},{"osd":3,"uuid":"37a1583c-4fb8-412a-a0c8-f83b1ff26a91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6811","nonce":2784619821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6813","nonce":2784619821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6817","nonce":2784619821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6815","nonce":2784619821}]},"public_addr":"172.21.15.17:6811/2784619821","cluster_addr":"172.21.15.17:6813/2784619821","heartbeat_back_addr":"172.21.15.17:6817/2784619821","heartbeat_front_addr":"172.21.15.17:6815/2784619821","state":["exists","up"]},{"osd":4,"uuid":"f4b9f377-d519-4b14-ac4a-f1debaeed9f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6809","nonce":3151801173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6811","nonce":3151801173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6815","nonce":3151801173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6813","nonce":3151801173}]},"public_addr":"172.21.15.93:6809/3151801173","cluster_addr":"172.21.15.93:6811/3151801173","heartbeat_back_addr":"172.21.15.93:6815/3151801173","heartbeat_front_addr":"172.21.15.93:6813/3151801173","state":["exists","up"]},{"osd":5,"uuid":"2d2af354-0535-4ef9-b166-3ef219498db8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6809","nonce":2184413924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6811","nonce":2184413924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6815","nonce":2184413924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6813","nonce":2184413924}]},"public_addr":"172.21.15.97:6809/2184413924","cluster_addr":"172.21.15.97:6811/2184413924","heartbeat_back_addr":"172.21.15.97:6815/2184413924","heartbeat_front_addr":"172.21.15.97:6813/2184413924","state":["exists","up"]},{"osd":6,"uuid":"9cdf0c4b-62dc-4168-864e-a9d1650a9ea8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6819","nonce":50486581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6821","nonce":50486581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6825","nonce":50486581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6823","nonce":50486581}]},"public_addr":"172.21.15.17:6819/50486581","cluster_addr":"172.21.15.17:6821/50486581","heartbeat_back_addr":"172.21.15.17:6825/50486581","heartbeat_front_addr":"172.21.15.17:6823/50486581","state":["exists","up"]},{"osd":7,"uuid":"09f60823-5491-45cd-9324-6ae0d56ce262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6817","nonce":2596912392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6819","nonce":2596912392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6823","nonce":2596912392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6821","nonce":2596912392}]},"public_addr":"172.21.15.93:6817/2596912392","cluster_addr":"172.21.15.93:6819/2596912392","heartbeat_back_addr":"172.21.15.93:6823/2596912392","heartbeat_front_addr":"172.21.15.93:6821/2596912392","state":["exists","up"]},{"osd":8,"uuid":"8c5771c9-718e-4313-a2b6-5e1c37c53f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6817","nonce":1965492859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6819","nonce":1965492859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6823","nonce":1965492859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6821","nonce":1965492859}]},"public_addr":"172.21.15.97:6817/1965492859","cluster_addr":"172.21.15.97:6819/1965492859","heartbeat_back_addr":"172.21.15.97:6823/1965492859","heartbeat_front_addr":"172.21.15.97:6821/1965492859","state":["exists","up"]},{"osd":9,"uuid":"1197b8a3-c09f-4c57-aba1-9e988a1620b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6827","nonce":2512645700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6829","nonce":2512645700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6833","nonce":2512645700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6831","nonce":2512645700}]},"public_addr":"172.21.15.17:6827/2512645700","cluster_addr":"172.21.15.17:6829/2512645700","heartbeat_back_addr":"172.21.15.17:6833/2512645700","heartbeat_front_addr":"172.21.15.17:6831/2512645700","state":["exists","up"]},{"osd":10,"uuid":"f99918c2-cdd8-4a7a-be2e-d1d6e0f828a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6825","nonce":264429558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6827","nonce":264429558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6831","nonce":264429558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6829","nonce":264429558}]},"public_addr":"172.21.15.93:6825/264429558","cluster_addr":"172.21.15.93:6827/264429558","heartbeat_back_addr":"172.21.15.93:6831/264429558","heartbeat_front_addr":"172.21.15.93:6829/264429558","state":["exists","up"]},{"osd":11,"uuid":"6bbb5b79-8d4d-441f-9583-203df6c78cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6825","nonce":3389913754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6827","nonce":3389913754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6831","nonce":3389913754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6829","nonce":3389913754}]},"public_addr":"172.21.15.97:6825/3389913754","cluster_addr":"172.21.15.97:6827/3389913754","heartbeat_back_addr":"172.21.15.97:6831/3389913754","heartbeat_front_addr":"172.21.15.97:6829/3389913754","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:54:41.079618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:03.207899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:24.136961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:50.392609+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:56:12.136237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:56:33.212314+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:00.243646+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:22.059234+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:43.232399+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:10.043896+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:31.910780+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:53.158983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.9","mappings":[{"from":5,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/2084033276":"2023-07-15T11:53:05.639597+0000","172.21.15.17:6801/1059284716":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/3651785636":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/1373538024":"2023-07-15T11:52:37.887946+0000","172.21.15.17:0/3560395020":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6800/1059284716":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/4262912861":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/2258375202":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6800/2804227742":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6801/2804227742":"2023-07-15T11:52:37.887946+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T12:00:12.261 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.282+0000 7f3c815ff700 1 -- 172.21.15.17:0/1713453134 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3c680a11d0 msgr2=0x7f3c680a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:12.261 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.282+0000 7f3c815ff700 1 --2- 172.21.15.17:0/1713453134 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3c680a11d0 0x7f3c680a3680 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c011550 tx=0x7f3c6c009260 comp rx=0 tx=0).stop 2023-07-14T12:00:12.262 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.282+0000 7f3c815ff700 1 -- 172.21.15.17:0/1713453134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c7c102f40 msgr2=0x7f3c7c197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:12.262 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.282+0000 7f3c815ff700 1 --2- 172.21.15.17:0/1713453134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c7c102f40 0x7f3c7c197ff0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3c6400b680 tx=0x7f3c64009580 comp rx=0 tx=0).stop 2023-07-14T12:00:12.262 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.282+0000 7f3c815ff700 1 -- 172.21.15.17:0/1713453134 shutdown_connections 2023-07-14T12:00:12.262 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.282+0000 7f3c815ff700 1 --2- 172.21.15.17:0/1713453134 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3c680a11d0 0x7f3c680a3680 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:12.262 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.282+0000 7f3c815ff700 1 --2- 172.21.15.17:0/1713453134 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c7c105860 0x7f3c7c19f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:12.262 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.282+0000 7f3c815ff700 1 --2- 172.21.15.17:0/1713453134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c7c100620 0x7f3c7c197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:12.262 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.282+0000 7f3c815ff700 1 --2- 172.21.15.17:0/1713453134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c7c102f40 0x7f3c7c197ff0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:12.262 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.282+0000 7f3c815ff700 1 -- 172.21.15.17:0/1713453134 >> 172.21.15.17:0/1713453134 conn(0x7f3c7c0fa120 msgr2=0x7f3c7c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:12.263 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.282+0000 7f3c815ff700 1 -- 172.21.15.17:0/1713453134 shutdown_connections 2023-07-14T12:00:12.263 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.282+0000 7f3c815ff700 1 -- 172.21.15.17:0/1713453134 wait complete. 2023-07-14T12:00:12.280 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-14T12:00:12.487 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.508+0000 7f5ee4bff700 1 -- 172.21.15.17:0/3134123635 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ee00ffbe0 msgr2=0x7f5ee00ffff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:12.488 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.508+0000 7f5ee4bff700 1 --2- 172.21.15.17:0/3134123635 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ee00ffbe0 0x7f5ee00ffff0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5ed00098f0 tx=0x7f5ed0009c00 comp rx=0 tx=0).stop 2023-07-14T12:00:12.488 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.508+0000 7f5ee4bff700 1 -- 172.21.15.17:0/3134123635 shutdown_connections 2023-07-14T12:00:12.488 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.508+0000 7f5ee4bff700 1 --2- 172.21.15.17:0/3134123635 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ee00ffbe0 0x7f5ee00ffff0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:12.488 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.508+0000 7f5ee4bff700 1 --2- 172.21.15.17:0/3134123635 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ee0101760 0x7f5ee0108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:12.488 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.508+0000 7f5ee4bff700 1 --2- 172.21.15.17:0/3134123635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ee0100dd0 0x7f5ee0101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:12.488 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.508+0000 7f5ee4bff700 1 -- 172.21.15.17:0/3134123635 >> 172.21.15.17:0/3134123635 conn(0x7f5ee00fb200 msgr2=0x7f5ee00fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:12.489 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.508+0000 7f5ee4bff700 1 -- 172.21.15.17:0/3134123635 shutdown_connections 2023-07-14T12:00:12.489 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.508+0000 7f5ee4bff700 1 -- 172.21.15.17:0/3134123635 wait complete. 2023-07-14T12:00:12.490 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.509+0000 7f5ee4bff700 1 Processor -- start 2023-07-14T12:00:12.490 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.509+0000 7f5ee4bff700 1 -- start start 2023-07-14T12:00:12.491 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.509+0000 7f5ee4bff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ee00ffbe0 0x7f5ee00742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:12.491 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.509+0000 7f5ee4bff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ee0100dd0 0x7f5ee0072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:12.491 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.509+0000 7f5ee4bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ee0101760 0x7f5ee0072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:12.491 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.509+0000 7f5ee4bff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee00735e0 con 0x7f5ee00ffbe0 2023-07-14T12:00:12.491 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.509+0000 7f5ee4bff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee0073750 con 0x7f5ee0100dd0 2023-07-14T12:00:12.491 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.509+0000 7f5ee4bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee019fa10 con 0x7f5ee0101760 2023-07-14T12:00:12.491 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.510+0000 7f5eddd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ee0100dd0 0x7f5ee0072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:12.492 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.510+0000 7f5ede59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ee00ffbe0 0x7f5ee00742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:12.492 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.510+0000 7f5eddd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ee0100dd0 0x7f5ee0072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.17:51594/0 (socket says 172.21.15.17:51594) 2023-07-14T12:00:12.492 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.510+0000 7f5eddd9b700 1 -- 172.21.15.17:0/3771899879 learned_addr learned my addr 172.21.15.17:0/3771899879 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:00:12.492 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.510+0000 7f5eded9d700 1 --2- 172.21.15.17:0/3771899879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ee0101760 0x7f5ee0072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:12.492 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.510+0000 7f5eddd9b700 1 -- 172.21.15.17:0/3771899879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ee0101760 msgr2=0x7f5ee0072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:12.492 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.510+0000 7f5eddd9b700 1 --2- 172.21.15.17:0/3771899879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ee0101760 0x7f5ee0072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:12.492 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.510+0000 7f5eddd9b700 1 -- 172.21.15.17:0/3771899879 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ee00ffbe0 msgr2=0x7f5ee00742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:12.493 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.510+0000 7f5eddd9b700 1 --2- 172.21.15.17:0/3771899879 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ee00ffbe0 0x7f5ee00742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:12.493 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.510+0000 7f5eddd9b700 1 -- 172.21.15.17:0/3771899879 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed8009d00 con 0x7f5ee0100dd0 2023-07-14T12:00:12.493 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.510+0000 7f5eddd9b700 1 --2- 172.21.15.17:0/3771899879 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ee0100dd0 0x7f5ee0072900 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5ed400e9a0 tx=0x7f5ed400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:12.493 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.510+0000 7f5ecf7fe700 1 -- 172.21.15.17:0/3771899879 <== mon.2 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed4009790 con 0x7f5ee0100dd0 2023-07-14T12:00:12.493 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.511+0000 7f5ee4bff700 1 -- 172.21.15.17:0/3771899879 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ed0009520 con 0x7f5ee0100dd0 2023-07-14T12:00:12.493 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.511+0000 7f5ecf7fe700 1 -- 172.21.15.17:0/3771899879 <== mon.2 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ed400cbd0 con 0x7f5ee0100dd0 2023-07-14T12:00:12.493 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.511+0000 7f5ecf7fe700 1 -- 172.21.15.17:0/3771899879 <== mon.2 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed4018680 con 0x7f5ee0100dd0 2023-07-14T12:00:12.493 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.511+0000 7f5ee4bff700 1 -- 172.21.15.17:0/3771899879 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ee01a00c0 con 0x7f5ee0100dd0 2023-07-14T12:00:12.494 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.512+0000 7f5ee4bff700 1 -- 172.21.15.17:0/3771899879 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ec0005320 con 0x7f5ee0100dd0 2023-07-14T12:00:12.494 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.514+0000 7f5ecf7fe700 1 -- 172.21.15.17:0/3771899879 <== mon.2 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5ed4010430 con 0x7f5ee0100dd0 2023-07-14T12:00:12.495 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.515+0000 7f5ecf7fe700 1 --2- 172.21.15.17:0/3771899879 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f5ec80a11d0 0x7f5ec80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:12.495 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.516+0000 7f5ecf7fe700 1 -- 172.21.15.17:0/3771899879 <== mon.2 v2:172.21.15.93:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5ed4014070 con 0x7f5ee0100dd0 2023-07-14T12:00:12.495 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.516+0000 7f5ede59c700 1 --2- 172.21.15.17:0/3771899879 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f5ec80a11d0 0x7f5ec80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:12.496 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.516+0000 7f5ede59c700 1 --2- 172.21.15.17:0/3771899879 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f5ec80a11d0 0x7f5ec80a3680 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0009fd0 tx=0x7f5ed0019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:12.496 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.517+0000 7f5ecf7fe700 1 -- 172.21.15.17:0/3771899879 <== mon.2 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5ed4051790 con 0x7f5ee0100dd0 2023-07-14T12:00:12.752 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.773+0000 7f5ee4bff700 1 -- 172.21.15.17:0/3771899879 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f5ec0005190 con 0x7f5ee0100dd0 2023-07-14T12:00:12.816 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.837+0000 7f5ecf7fe700 1 -- 172.21.15.17:0/3771899879 <== mon.2 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7f5ed4096270 con 0x7f5ee0100dd0 2023-07-14T12:00:12.819 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.839+0000 7f5ee4bff700 1 -- 172.21.15.17:0/3771899879 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f5ec80a11d0 msgr2=0x7f5ec80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:12.819 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.839+0000 7f5ee4bff700 1 --2- 172.21.15.17:0/3771899879 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f5ec80a11d0 0x7f5ec80a3680 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0009fd0 tx=0x7f5ed0019040 comp rx=0 tx=0).stop 2023-07-14T12:00:12.819 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.839+0000 7f5ee4bff700 1 -- 172.21.15.17:0/3771899879 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ee0100dd0 msgr2=0x7f5ee0072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:12.819 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.839+0000 7f5ee4bff700 1 --2- 172.21.15.17:0/3771899879 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ee0100dd0 0x7f5ee0072900 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5ed400e9a0 tx=0x7f5ed400ecb0 comp rx=0 tx=0).stop 2023-07-14T12:00:12.819 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.839+0000 7f5ee4bff700 1 -- 172.21.15.17:0/3771899879 shutdown_connections 2023-07-14T12:00:12.820 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.839+0000 7f5ee4bff700 1 --2- 172.21.15.17:0/3771899879 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f5ec80a11d0 0x7f5ec80a3680 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:12.820 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.839+0000 7f5ee4bff700 1 --2- 172.21.15.17:0/3771899879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ee0101760 0x7f5ee0072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:12.820 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.839+0000 7f5ee4bff700 1 --2- 172.21.15.17:0/3771899879 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ee00ffbe0 0x7f5ee00742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:12.820 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.839+0000 7f5ee4bff700 1 --2- 172.21.15.17:0/3771899879 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ee0100dd0 0x7f5ee0072900 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:12.820 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.839+0000 7f5ee4bff700 1 -- 172.21.15.17:0/3771899879 >> 172.21.15.17:0/3771899879 conn(0x7f5ee00fb200 msgr2=0x7f5ee00fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:12.820 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.839+0000 7f5ee4bff700 1 -- 172.21.15.17:0/3771899879 shutdown_connections 2023-07-14T12:00:12.820 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:12.840+0000 7f5ee4bff700 1 -- 172.21.15.17:0/3771899879 wait complete. 2023-07-14T12:00:12.838 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True} 2023-07-14T12:00:12.838 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2023-07-14T12:00:13.047 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.067+0000 7f8a54fff700 1 -- 172.21.15.17:0/1961911285 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a50100dd0 msgr2=0x7f8a50101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:13.047 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.067+0000 7f8a54fff700 1 --2- 172.21.15.17:0/1961911285 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a50100dd0 0x7f8a50101220 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8a44009990 tx=0x7f8a44009ca0 comp rx=0 tx=0).stop 2023-07-14T12:00:13.047 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.067+0000 7f8a54fff700 1 -- 172.21.15.17:0/1961911285 shutdown_connections 2023-07-14T12:00:13.047 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.067+0000 7f8a54fff700 1 --2- 172.21.15.17:0/1961911285 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8a500ffbe0 0x7f8a500ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:13.047 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.067+0000 7f8a54fff700 1 --2- 172.21.15.17:0/1961911285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a50101760 0x7f8a50108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:13.047 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.067+0000 7f8a54fff700 1 --2- 172.21.15.17:0/1961911285 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a50100dd0 0x7f8a50101220 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:13.048 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.067+0000 7f8a54fff700 1 -- 172.21.15.17:0/1961911285 >> 172.21.15.17:0/1961911285 conn(0x7f8a500fb200 msgr2=0x7f8a500fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:13.048 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.067+0000 7f8a54fff700 1 -- 172.21.15.17:0/1961911285 shutdown_connections 2023-07-14T12:00:13.048 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.067+0000 7f8a54fff700 1 -- 172.21.15.17:0/1961911285 wait complete. 2023-07-14T12:00:13.049 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.068+0000 7f8a54fff700 1 Processor -- start 2023-07-14T12:00:13.049 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.068+0000 7f8a54fff700 1 -- start start 2023-07-14T12:00:13.050 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.069+0000 7f8a54fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8a500ffbe0 0x7f8a50197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:13.050 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.069+0000 7f8a54fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a50100dd0 0x7f8a501980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:13.050 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.069+0000 7f8a54fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a50101760 0x7f8a5019f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:13.050 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.069+0000 7f8a54fff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5019f730 con 0x7f8a50100dd0 2023-07-14T12:00:13.050 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.069+0000 7f8a54fff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5019f870 con 0x7f8a500ffbe0 2023-07-14T12:00:13.050 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.069+0000 7f8a54fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5019fa00 con 0x7f8a50101760 2023-07-14T12:00:13.050 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.069+0000 7f8a4dd9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a50100dd0 0x7f8a501980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:13.051 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.069+0000 7f8a4dd9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a50100dd0 0x7f8a501980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44860/0 (socket says 172.21.15.17:44860) 2023-07-14T12:00:13.051 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.069+0000 7f8a4dd9b700 1 -- 172.21.15.17:0/2492381480 learned_addr learned my addr 172.21.15.17:0/2492381480 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:00:13.051 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.069+0000 7f8a4e59c700 1 --2- 172.21.15.17:0/2492381480 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8a500ffbe0 0x7f8a50197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:13.051 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.069+0000 7f8a4ed9d700 1 --2- 172.21.15.17:0/2492381480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a50101760 0x7f8a5019f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:13.051 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.069+0000 7f8a4dd9b700 1 -- 172.21.15.17:0/2492381480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a50101760 msgr2=0x7f8a5019f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:13.051 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.069+0000 7f8a4dd9b700 1 --2- 172.21.15.17:0/2492381480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a50101760 0x7f8a5019f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:13.051 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.069+0000 7f8a4dd9b700 1 -- 172.21.15.17:0/2492381480 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8a500ffbe0 msgr2=0x7f8a50197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:13.052 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.069+0000 7f8a4dd9b700 1 --2- 172.21.15.17:0/2492381480 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8a500ffbe0 0x7f8a50197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:13.052 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.069+0000 7f8a4dd9b700 1 -- 172.21.15.17:0/2492381480 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a440095f0 con 0x7f8a50100dd0 2023-07-14T12:00:13.052 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.070+0000 7f8a4dd9b700 1 --2- 172.21.15.17:0/2492381480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a50100dd0 0x7f8a501980f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8a44004760 tx=0x7f8a44004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:13.052 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.070+0000 7f8a3f7fe700 1 -- 172.21.15.17:0/2492381480 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4401d070 con 0x7f8a50100dd0 2023-07-14T12:00:13.052 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.070+0000 7f8a3f7fe700 1 -- 172.21.15.17:0/2492381480 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a44004830 con 0x7f8a50100dd0 2023-07-14T12:00:13.052 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.070+0000 7f8a54fff700 1 -- 172.21.15.17:0/2492381480 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a5019fc80 con 0x7f8a50100dd0 2023-07-14T12:00:13.052 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.070+0000 7f8a3f7fe700 1 -- 172.21.15.17:0/2492381480 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a44005350 con 0x7f8a50100dd0 2023-07-14T12:00:13.052 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.070+0000 7f8a54fff700 1 -- 172.21.15.17:0/2492381480 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a501a0040 con 0x7f8a50100dd0 2023-07-14T12:00:13.053 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.071+0000 7f8a54fff700 1 -- 172.21.15.17:0/2492381480 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a5010b400 con 0x7f8a50100dd0 2023-07-14T12:00:13.054 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.075+0000 7f8a3f7fe700 1 -- 172.21.15.17:0/2492381480 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8a440054d0 con 0x7f8a50100dd0 2023-07-14T12:00:13.055 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.076+0000 7f8a3f7fe700 1 --2- 172.21.15.17:0/2492381480 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f8a380a1150 0x7f8a380a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:13.055 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.076+0000 7f8a3f7fe700 1 -- 172.21.15.17:0/2492381480 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8a440ce320 con 0x7f8a50100dd0 2023-07-14T12:00:13.056 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.076+0000 7f8a4e59c700 1 --2- 172.21.15.17:0/2492381480 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f8a380a1150 0x7f8a380a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:13.056 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.077+0000 7f8a4e59c700 1 --2- 172.21.15.17:0/2492381480 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f8a380a1150 0x7f8a380a3600 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8a50100c30 tx=0x7f8a40009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:13.057 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.078+0000 7f8a3f7fe700 1 -- 172.21.15.17:0/2492381480 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8a44097790 con 0x7f8a50100dd0 2023-07-14T12:00:13.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:12 smithi093 ceph-mon[128922]: pgmap v293: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2023-07-14T12:00:13.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:12 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/1602635014' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T12:00:13.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:12 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/1713453134' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T12:00:13.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:12 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/3771899879' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-14T12:00:13.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:12 smithi093 ceph-mon[128922]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-14T12:00:13.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:12 smithi097 ceph-mon[129313]: pgmap v293: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2023-07-14T12:00:13.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:12 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/1602635014' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T12:00:13.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:12 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/1713453134' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T12:00:13.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:12 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/3771899879' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-14T12:00:13.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:12 smithi097 ceph-mon[129313]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-14T12:00:13.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:12 smithi017 ceph-mon[126339]: pgmap v293: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2023-07-14T12:00:13.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:12 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/1602635014' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T12:00:13.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:12 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/1713453134' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T12:00:13.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:12 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/3771899879' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-14T12:00:13.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:12 smithi017 ceph-mon[126339]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-14T12:00:13.313 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.334+0000 7f8a54fff700 1 -- 172.21.15.17:0/2492381480 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7f8a501a05f0 con 0x7f8a50100dd0 2023-07-14T12:00:13.434 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.454+0000 7f8a3f7fe700 1 -- 172.21.15.17:0/2492381480 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7f8a4409adb0 con 0x7f8a50100dd0 2023-07-14T12:00:13.436 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.457+0000 7f8a54fff700 1 -- 172.21.15.17:0/2492381480 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f8a380a1150 msgr2=0x7f8a380a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:13.437 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.457+0000 7f8a54fff700 1 --2- 172.21.15.17:0/2492381480 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f8a380a1150 0x7f8a380a3600 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8a50100c30 tx=0x7f8a40009260 comp rx=0 tx=0).stop 2023-07-14T12:00:13.437 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.457+0000 7f8a54fff700 1 -- 172.21.15.17:0/2492381480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a50100dd0 msgr2=0x7f8a501980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:13.437 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.457+0000 7f8a54fff700 1 --2- 172.21.15.17:0/2492381480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a50100dd0 0x7f8a501980f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8a44004760 tx=0x7f8a44004790 comp rx=0 tx=0).stop 2023-07-14T12:00:13.437 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.457+0000 7f8a54fff700 1 -- 172.21.15.17:0/2492381480 shutdown_connections 2023-07-14T12:00:13.438 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.457+0000 7f8a54fff700 1 --2- 172.21.15.17:0/2492381480 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f8a380a1150 0x7f8a380a3600 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:13.438 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.457+0000 7f8a54fff700 1 --2- 172.21.15.17:0/2492381480 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8a500ffbe0 0x7f8a50197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:13.438 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.457+0000 7f8a54fff700 1 --2- 172.21.15.17:0/2492381480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a50101760 0x7f8a5019f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:13.439 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.457+0000 7f8a54fff700 1 --2- 172.21.15.17:0/2492381480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a50100dd0 0x7f8a501980f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:13.439 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.457+0000 7f8a54fff700 1 -- 172.21.15.17:0/2492381480 >> 172.21.15.17:0/2492381480 conn(0x7f8a500fb200 msgr2=0x7f8a50106660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:13.439 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.457+0000 7f8a54fff700 1 -- 172.21.15.17:0/2492381480 shutdown_connections 2023-07-14T12:00:13.440 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.457+0000 7f8a54fff700 1 -- 172.21.15.17:0/2492381480 wait complete. 2023-07-14T12:00:13.459 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-07-14T12:00:13.669 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.687+0000 7f36375ff700 1 -- 172.21.15.17:0/163966084 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3630100dd0 msgr2=0x7f3630101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:13.669 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.687+0000 7f36375ff700 1 --2- 172.21.15.17:0/163966084 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3630100dd0 0x7f3630101220 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3620009990 tx=0x7f3620009ca0 comp rx=0 tx=0).stop 2023-07-14T12:00:13.669 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.687+0000 7f36375ff700 1 -- 172.21.15.17:0/163966084 shutdown_connections 2023-07-14T12:00:13.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.687+0000 7f36375ff700 1 --2- 172.21.15.17:0/163966084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3630101760 0x7f3630108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:13.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.687+0000 7f36375ff700 1 --2- 172.21.15.17:0/163966084 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36300ffbe0 0x7f36300ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:13.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.687+0000 7f36375ff700 1 --2- 172.21.15.17:0/163966084 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3630100dd0 0x7f3630101220 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:13.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.687+0000 7f36375ff700 1 -- 172.21.15.17:0/163966084 >> 172.21.15.17:0/163966084 conn(0x7f36300fb200 msgr2=0x7f36300fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:13.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.687+0000 7f36375ff700 1 -- 172.21.15.17:0/163966084 shutdown_connections 2023-07-14T12:00:13.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.688+0000 7f36375ff700 1 -- 172.21.15.17:0/163966084 wait complete. 2023-07-14T12:00:13.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.688+0000 7f36375ff700 1 Processor -- start 2023-07-14T12:00:13.671 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.689+0000 7f36375ff700 1 -- start start 2023-07-14T12:00:13.671 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.689+0000 7f36375ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f36300ffbe0 0x7f3630197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:13.671 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.689+0000 7f36375ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3630100dd0 0x7f3630198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:13.671 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.689+0000 7f36375ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3630101760 0x7f363019f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:13.671 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.689+0000 7f36375ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363019f650 con 0x7f3630100dd0 2023-07-14T12:00:13.671 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.689+0000 7f36375ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363019f790 con 0x7f36300ffbe0 2023-07-14T12:00:13.671 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.689+0000 7f36375ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363019f920 con 0x7f3630101760 2023-07-14T12:00:13.672 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.689+0000 7f3634b9a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3630100dd0 0x7f3630198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:13.672 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.689+0000 7f3634b9a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3630100dd0 0x7f3630198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44882/0 (socket says 172.21.15.17:44882) 2023-07-14T12:00:13.672 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.689+0000 7f3634b9a700 1 -- 172.21.15.17:0/2568918624 learned_addr learned my addr 172.21.15.17:0/2568918624 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:00:13.672 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.689+0000 7f363539b700 1 --2- 172.21.15.17:0/2568918624 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f36300ffbe0 0x7f3630197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:13.672 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.690+0000 7f3635b9c700 1 --2- 172.21.15.17:0/2568918624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3630101760 0x7f363019f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:13.672 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.690+0000 7f3634b9a700 1 -- 172.21.15.17:0/2568918624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3630101760 msgr2=0x7f363019f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:13.673 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.690+0000 7f3634b9a700 1 --2- 172.21.15.17:0/2568918624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3630101760 0x7f363019f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:13.673 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.690+0000 7f3634b9a700 1 -- 172.21.15.17:0/2568918624 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f36300ffbe0 msgr2=0x7f3630197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:13.673 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.690+0000 7f3634b9a700 1 --2- 172.21.15.17:0/2568918624 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f36300ffbe0 0x7f3630197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:13.673 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.690+0000 7f3634b9a700 1 -- 172.21.15.17:0/2568918624 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f362c009520 con 0x7f3630100dd0 2023-07-14T12:00:13.673 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.690+0000 7f3635b9c700 1 --2- 172.21.15.17:0/2568918624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3630101760 0x7f363019f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T12:00:13.674 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.690+0000 7f3634b9a700 1 --2- 172.21.15.17:0/2568918624 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3630100dd0 0x7f3630198010 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3620005740 tx=0x7f3620005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:13.674 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.690+0000 7f36267fc700 1 -- 172.21.15.17:0/2568918624 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362001d070 con 0x7f3630100dd0 2023-07-14T12:00:13.674 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.690+0000 7f36375ff700 1 -- 172.21.15.17:0/2568918624 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36200095f0 con 0x7f3630100dd0 2023-07-14T12:00:13.675 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.690+0000 7f36267fc700 1 -- 172.21.15.17:0/2568918624 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f362000fbf0 con 0x7f3630100dd0 2023-07-14T12:00:13.675 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.690+0000 7f36267fc700 1 -- 172.21.15.17:0/2568918624 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36200173f0 con 0x7f3630100dd0 2023-07-14T12:00:13.675 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.691+0000 7f36375ff700 1 -- 172.21.15.17:0/2568918624 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f363019ff10 con 0x7f3630100dd0 2023-07-14T12:00:13.675 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.692+0000 7f36375ff700 1 -- 172.21.15.17:0/2568918624 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f363010b3b0 con 0x7f3630100dd0 2023-07-14T12:00:13.675 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.694+0000 7f36267fc700 1 -- 172.21.15.17:0/2568918624 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3620021410 con 0x7f3630100dd0 2023-07-14T12:00:13.676 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.695+0000 7f36267fc700 1 --2- 172.21.15.17:0/2568918624 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f361c0a1150 0x7f361c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:13.676 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.695+0000 7f36267fc700 1 -- 172.21.15.17:0/2568918624 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f36200cdde0 con 0x7f3630100dd0 2023-07-14T12:00:13.676 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.695+0000 7f363539b700 1 --2- 172.21.15.17:0/2568918624 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f361c0a1150 0x7f361c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:13.677 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.696+0000 7f363539b700 1 --2- 172.21.15.17:0/2568918624 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f361c0a1150 0x7f361c0a3600 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3630100c30 tx=0x7f362c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:13.678 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.699+0000 7f36267fc700 1 -- 172.21.15.17:0/2568918624 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3620097370 con 0x7f3630100dd0 2023-07-14T12:00:13.935 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:13.955+0000 7f36375ff700 1 -- 172.21.15.17:0/2568918624 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f36301a0740 con 0x7f3630100dd0 2023-07-14T12:00:14.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:13 smithi093 ceph-mon[128922]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-07-14T12:00:14.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:13 smithi093 ceph-mon[128922]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2023-07-14T12:00:14.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:13 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/2492381480' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-07-14T12:00:14.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:13 smithi093 ceph-mon[128922]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-07-14T12:00:14.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:13 smithi093 ceph-mon[128922]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-07-14T12:00:14.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:13 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/2492381480' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-07-14T12:00:14.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:13 smithi093 ceph-mon[128922]: fsmap cephfs:2 {0=a=up:active,1=b=up:creating} 10 up:standby 2023-07-14T12:00:14.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:13 smithi093 ceph-mon[128922]: daemon mds.b is now active in filesystem cephfs as rank 1 2023-07-14T12:00:14.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:13 smithi097 ceph-mon[129313]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-07-14T12:00:14.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:13 smithi097 ceph-mon[129313]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2023-07-14T12:00:14.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:13 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/2492381480' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-07-14T12:00:14.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:13 smithi097 ceph-mon[129313]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-07-14T12:00:14.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:13 smithi097 ceph-mon[129313]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-07-14T12:00:14.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:13 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/2492381480' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-07-14T12:00:14.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:13 smithi097 ceph-mon[129313]: fsmap cephfs:2 {0=a=up:active,1=b=up:creating} 10 up:standby 2023-07-14T12:00:14.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:13 smithi097 ceph-mon[129313]: daemon mds.b is now active in filesystem cephfs as rank 1 2023-07-14T12:00:14.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:13 smithi017 ceph-mon[126339]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-07-14T12:00:14.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:13 smithi017 ceph-mon[126339]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2023-07-14T12:00:14.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:13 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/2492381480' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-07-14T12:00:14.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:13 smithi017 ceph-mon[126339]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-07-14T12:00:14.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:13 smithi017 ceph-mon[126339]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-07-14T12:00:14.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:13 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/2492381480' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-07-14T12:00:14.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:13 smithi017 ceph-mon[126339]: fsmap cephfs:2 {0=a=up:active,1=b=up:creating} 10 up:standby 2023-07-14T12:00:14.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:13 smithi017 ceph-mon[126339]: daemon mds.b is now active in filesystem cephfs as rank 1 2023-07-14T12:00:14.445 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.466+0000 7f36267fc700 1 -- 172.21.15.17:0/2568918624 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7f362009a990 con 0x7f3630100dd0 2023-07-14T12:00:14.449 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.470+0000 7f36375ff700 1 -- 172.21.15.17:0/2568918624 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f361c0a1150 msgr2=0x7f361c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:14.450 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.470+0000 7f36375ff700 1 --2- 172.21.15.17:0/2568918624 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f361c0a1150 0x7f361c0a3600 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3630100c30 tx=0x7f362c009260 comp rx=0 tx=0).stop 2023-07-14T12:00:14.450 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.470+0000 7f36375ff700 1 -- 172.21.15.17:0/2568918624 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3630100dd0 msgr2=0x7f3630198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:14.450 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.470+0000 7f36375ff700 1 --2- 172.21.15.17:0/2568918624 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3630100dd0 0x7f3630198010 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3620005740 tx=0x7f3620005770 comp rx=0 tx=0).stop 2023-07-14T12:00:14.450 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.470+0000 7f36375ff700 1 -- 172.21.15.17:0/2568918624 shutdown_connections 2023-07-14T12:00:14.450 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.470+0000 7f36375ff700 1 --2- 172.21.15.17:0/2568918624 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f361c0a1150 0x7f361c0a3600 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:14.451 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.470+0000 7f36375ff700 1 --2- 172.21.15.17:0/2568918624 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f36300ffbe0 0x7f3630197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:14.451 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.470+0000 7f36375ff700 1 --2- 172.21.15.17:0/2568918624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3630101760 0x7f363019f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:14.451 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.470+0000 7f36375ff700 1 --2- 172.21.15.17:0/2568918624 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3630100dd0 0x7f3630198010 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:14.451 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.470+0000 7f36375ff700 1 -- 172.21.15.17:0/2568918624 >> 172.21.15.17:0/2568918624 conn(0x7f36300fb200 msgr2=0x7f3630106650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:14.451 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.470+0000 7f36375ff700 1 -- 172.21.15.17:0/2568918624 shutdown_connections 2023-07-14T12:00:14.451 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.470+0000 7f36375ff700 1 -- 172.21.15.17:0/2568918624 wait complete. 2023-07-14T12:00:14.474 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-07-14T12:00:14.684 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.704+0000 7f7c613ff700 1 -- 172.21.15.17:0/951456056 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c5c100dd0 msgr2=0x7f7c5c101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:14.685 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.705+0000 7f7c613ff700 1 --2- 172.21.15.17:0/951456056 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c5c100dd0 0x7f7c5c101220 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7c50009990 tx=0x7f7c50009ca0 comp rx=0 tx=0).stop 2023-07-14T12:00:14.685 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.705+0000 7f7c613ff700 1 -- 172.21.15.17:0/951456056 shutdown_connections 2023-07-14T12:00:14.685 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.705+0000 7f7c613ff700 1 --2- 172.21.15.17:0/951456056 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7c5c101760 0x7f7c5c108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:14.685 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.705+0000 7f7c613ff700 1 --2- 172.21.15.17:0/951456056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c5c0ffbe0 0x7f7c5c0ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:14.685 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.705+0000 7f7c613ff700 1 --2- 172.21.15.17:0/951456056 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c5c100dd0 0x7f7c5c101220 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:14.686 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.705+0000 7f7c613ff700 1 -- 172.21.15.17:0/951456056 >> 172.21.15.17:0/951456056 conn(0x7f7c5c0fb200 msgr2=0x7f7c5c0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:14.686 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.705+0000 7f7c613ff700 1 -- 172.21.15.17:0/951456056 shutdown_connections 2023-07-14T12:00:14.686 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.705+0000 7f7c613ff700 1 -- 172.21.15.17:0/951456056 wait complete. 2023-07-14T12:00:14.687 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.706+0000 7f7c613ff700 1 Processor -- start 2023-07-14T12:00:14.687 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.706+0000 7f7c613ff700 1 -- start start 2023-07-14T12:00:14.688 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.706+0000 7f7c613ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c5c0ffbe0 0x7f7c5c0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:14.688 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.706+0000 7f7c613ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c5c100dd0 0x7f7c5c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:14.688 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.706+0000 7f7c613ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7c5c101760 0x7f7c5c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:14.688 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.706+0000 7f7c613ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5c0735e0 con 0x7f7c5c100dd0 2023-07-14T12:00:14.688 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.706+0000 7f7c613ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5c073750 con 0x7f7c5c101760 2023-07-14T12:00:14.688 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.706+0000 7f7c613ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5c19f990 con 0x7f7c5c0ffbe0 2023-07-14T12:00:14.689 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.707+0000 7f7c5a7fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c5c100dd0 0x7f7c5c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:14.689 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.707+0000 7f7c5a7fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c5c100dd0 0x7f7c5c072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44906/0 (socket says 172.21.15.17:44906) 2023-07-14T12:00:14.689 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.707+0000 7f7c5a7fc700 1 -- 172.21.15.17:0/3133303889 learned_addr learned my addr 172.21.15.17:0/3133303889 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:00:14.689 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.707+0000 7f7c5b7fe700 1 --2- 172.21.15.17:0/3133303889 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7c5c101760 0x7f7c5c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:14.689 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.707+0000 7f7c5affd700 1 --2- 172.21.15.17:0/3133303889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c5c0ffbe0 0x7f7c5c0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:14.689 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.707+0000 7f7c5a7fc700 1 -- 172.21.15.17:0/3133303889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c5c0ffbe0 msgr2=0x7f7c5c0742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:14.690 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.707+0000 7f7c5a7fc700 1 --2- 172.21.15.17:0/3133303889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c5c0ffbe0 0x7f7c5c0742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:14.690 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.707+0000 7f7c5a7fc700 1 -- 172.21.15.17:0/3133303889 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7c5c101760 msgr2=0x7f7c5c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:14.690 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.707+0000 7f7c5a7fc700 1 --2- 172.21.15.17:0/3133303889 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7c5c101760 0x7f7c5c072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:14.690 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.707+0000 7f7c5a7fc700 1 -- 172.21.15.17:0/3133303889 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c500095f0 con 0x7f7c5c100dd0 2023-07-14T12:00:14.690 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.707+0000 7f7c5a7fc700 1 --2- 172.21.15.17:0/3133303889 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c5c100dd0 0x7f7c5c072900 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7c50004760 tx=0x7f7c50004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:14.690 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.708+0000 7f7c43fff700 1 -- 172.21.15.17:0/3133303889 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5001d070 con 0x7f7c5c100dd0 2023-07-14T12:00:14.690 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.708+0000 7f7c43fff700 1 -- 172.21.15.17:0/3133303889 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c50004830 con 0x7f7c5c100dd0 2023-07-14T12:00:14.691 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.708+0000 7f7c613ff700 1 -- 172.21.15.17:0/3133303889 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c5c19fc10 con 0x7f7c5c100dd0 2023-07-14T12:00:14.691 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.708+0000 7f7c43fff700 1 -- 172.21.15.17:0/3133303889 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c50005350 con 0x7f7c5c100dd0 2023-07-14T12:00:14.691 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.708+0000 7f7c613ff700 1 -- 172.21.15.17:0/3133303889 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c5c1a0030 con 0x7f7c5c100dd0 2023-07-14T12:00:14.692 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.709+0000 7f7c613ff700 1 -- 172.21.15.17:0/3133303889 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c5c10b400 con 0x7f7c5c100dd0 2023-07-14T12:00:14.692 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.711+0000 7f7c43fff700 1 -- 172.21.15.17:0/3133303889 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7c50005530 con 0x7f7c5c100dd0 2023-07-14T12:00:14.692 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.712+0000 7f7c43fff700 1 --2- 172.21.15.17:0/3133303889 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f7c440a11d0 0x7f7c440a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:14.692 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.712+0000 7f7c43fff700 1 -- 172.21.15.17:0/3133303889 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7c500cddb0 con 0x7f7c5c100dd0 2023-07-14T12:00:14.692 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.713+0000 7f7c5affd700 1 --2- 172.21.15.17:0/3133303889 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f7c440a11d0 0x7f7c440a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:14.693 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.713+0000 7f7c5affd700 1 --2- 172.21.15.17:0/3133303889 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f7c440a11d0 0x7f7c440a3680 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c100c30 tx=0x7f7c4c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:14.695 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.716+0000 7f7c43fff700 1 -- 172.21.15.17:0/3133303889 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7c5005a790 con 0x7f7c5c100dd0 2023-07-14T12:00:14.962 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:14.982+0000 7f7c613ff700 1 -- 172.21.15.17:0/3133303889 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f7c5c106a90 con 0x7f7c5c100dd0 2023-07-14T12:00:15.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:14 smithi093 ceph-mon[128922]: pgmap v294: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 1 op/s 2023-07-14T12:00:15.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:14 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/2568918624' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-07-14T12:00:15.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:14 smithi093 ceph-mon[128922]: mds.? [v2:172.21.15.93:6838/1281342428,v1:172.21.15.93:6839/1281342428] up:active 2023-07-14T12:00:15.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:14 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/2568918624' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-07-14T12:00:15.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:14 smithi093 ceph-mon[128922]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-07-14T12:00:15.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:14 smithi093 ceph-mon[128922]: fsmap cephfs:2 {0=a=up:active,1=b=up:active} 10 up:standby 2023-07-14T12:00:15.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:14 smithi093 ceph-mon[128922]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:creating} 9 up:standby 2023-07-14T12:00:15.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:14 smithi093 ceph-mon[128922]: daemon mds.i is now active in filesystem cephfs as rank 2 2023-07-14T12:00:15.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:14 smithi097 ceph-mon[129313]: pgmap v294: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 1 op/s 2023-07-14T12:00:15.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:14 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/2568918624' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-07-14T12:00:15.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:14 smithi097 ceph-mon[129313]: mds.? [v2:172.21.15.93:6838/1281342428,v1:172.21.15.93:6839/1281342428] up:active 2023-07-14T12:00:15.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:14 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/2568918624' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-07-14T12:00:15.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:14 smithi097 ceph-mon[129313]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-07-14T12:00:15.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:14 smithi097 ceph-mon[129313]: fsmap cephfs:2 {0=a=up:active,1=b=up:active} 10 up:standby 2023-07-14T12:00:15.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:14 smithi097 ceph-mon[129313]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:creating} 9 up:standby 2023-07-14T12:00:15.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:14 smithi097 ceph-mon[129313]: daemon mds.i is now active in filesystem cephfs as rank 2 2023-07-14T12:00:15.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:14 smithi017 ceph-mon[126339]: pgmap v294: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 1 op/s 2023-07-14T12:00:15.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:14 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/2568918624' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-07-14T12:00:15.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:14 smithi017 ceph-mon[126339]: mds.? [v2:172.21.15.93:6838/1281342428,v1:172.21.15.93:6839/1281342428] up:active 2023-07-14T12:00:15.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:14 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/2568918624' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-07-14T12:00:15.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:14 smithi017 ceph-mon[126339]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-07-14T12:00:15.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:14 smithi017 ceph-mon[126339]: fsmap cephfs:2 {0=a=up:active,1=b=up:active} 10 up:standby 2023-07-14T12:00:15.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:14 smithi017 ceph-mon[126339]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:creating} 9 up:standby 2023-07-14T12:00:15.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:14 smithi017 ceph-mon[126339]: daemon mds.i is now active in filesystem cephfs as rank 2 2023-07-14T12:00:15.454 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.473+0000 7f7c43fff700 1 -- 172.21.15.17:0/3133303889 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7f7c50097220 con 0x7f7c5c100dd0 2023-07-14T12:00:15.456 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.477+0000 7f7c613ff700 1 -- 172.21.15.17:0/3133303889 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f7c440a11d0 msgr2=0x7f7c440a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:15.456 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.477+0000 7f7c613ff700 1 --2- 172.21.15.17:0/3133303889 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f7c440a11d0 0x7f7c440a3680 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c100c30 tx=0x7f7c4c009260 comp rx=0 tx=0).stop 2023-07-14T12:00:15.457 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.477+0000 7f7c613ff700 1 -- 172.21.15.17:0/3133303889 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c5c100dd0 msgr2=0x7f7c5c072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:15.457 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.477+0000 7f7c613ff700 1 --2- 172.21.15.17:0/3133303889 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c5c100dd0 0x7f7c5c072900 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7c50004760 tx=0x7f7c50004790 comp rx=0 tx=0).stop 2023-07-14T12:00:15.457 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.477+0000 7f7c613ff700 1 -- 172.21.15.17:0/3133303889 shutdown_connections 2023-07-14T12:00:15.457 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.477+0000 7f7c613ff700 1 --2- 172.21.15.17:0/3133303889 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f7c440a11d0 0x7f7c440a3680 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:15.458 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.477+0000 7f7c613ff700 1 --2- 172.21.15.17:0/3133303889 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7c5c101760 0x7f7c5c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:15.458 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.477+0000 7f7c613ff700 1 --2- 172.21.15.17:0/3133303889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c5c0ffbe0 0x7f7c5c0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:15.458 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.477+0000 7f7c613ff700 1 --2- 172.21.15.17:0/3133303889 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c5c100dd0 0x7f7c5c072900 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:15.458 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.477+0000 7f7c613ff700 1 -- 172.21.15.17:0/3133303889 >> 172.21.15.17:0/3133303889 conn(0x7f7c5c0fb200 msgr2=0x7f7c5c0fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:15.458 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.477+0000 7f7c613ff700 1 -- 172.21.15.17:0/3133303889 shutdown_connections 2023-07-14T12:00:15.458 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.477+0000 7f7c613ff700 1 -- 172.21.15.17:0/3133303889 wait complete. 2023-07-14T12:00:15.484 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-07-14T12:00:15.485 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T12:00:15.694 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.714+0000 7f26537ff700 1 -- 172.21.15.17:0/489185289 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f264c100e30 msgr2=0x7f264c1012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:15.694 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.714+0000 7f26537ff700 1 --2- 172.21.15.17:0/489185289 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f264c100e30 0x7f264c1012a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f263c009990 tx=0x7f263c009ca0 comp rx=0 tx=0).stop 2023-07-14T12:00:15.694 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.714+0000 7f26537ff700 1 -- 172.21.15.17:0/489185289 shutdown_connections 2023-07-14T12:00:15.695 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.714+0000 7f26537ff700 1 --2- 172.21.15.17:0/489185289 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f264c1017e0 0x7f264c108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:15.695 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.714+0000 7f26537ff700 1 --2- 172.21.15.17:0/489185289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f264c0ffba0 0x7f264c0fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:15.695 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.714+0000 7f26537ff700 1 --2- 172.21.15.17:0/489185289 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f264c100e30 0x7f264c1012a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:15.695 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.714+0000 7f26537ff700 1 -- 172.21.15.17:0/489185289 >> 172.21.15.17:0/489185289 conn(0x7f264c0fb1c0 msgr2=0x7f264c0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:15.695 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.715+0000 7f26537ff700 1 -- 172.21.15.17:0/489185289 shutdown_connections 2023-07-14T12:00:15.695 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.715+0000 7f26537ff700 1 -- 172.21.15.17:0/489185289 wait complete. 2023-07-14T12:00:15.697 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.715+0000 7f26537ff700 1 Processor -- start 2023-07-14T12:00:15.697 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.716+0000 7f26537ff700 1 -- start start 2023-07-14T12:00:15.697 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.716+0000 7f26537ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f264c0ffba0 0x7f264c197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:15.697 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.716+0000 7f26537ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f264c100e30 0x7f264c198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:15.697 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.716+0000 7f26537ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f264c1017e0 0x7f264c19f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:15.698 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.716+0000 7f26537ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264c19f6c0 con 0x7f264c1017e0 2023-07-14T12:00:15.698 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.716+0000 7f26537ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264c19f800 con 0x7f264c100e30 2023-07-14T12:00:15.698 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.716+0000 7f26537ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264c19f990 con 0x7f264c0ffba0 2023-07-14T12:00:15.698 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.716+0000 7f2651d9c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f264c1017e0 0x7f264c19f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:15.698 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.716+0000 7f2651d9c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f264c1017e0 0x7f264c19f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44924/0 (socket says 172.21.15.17:44924) 2023-07-14T12:00:15.698 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.716+0000 7f2651d9c700 1 -- 172.21.15.17:0/278934150 learned_addr learned my addr 172.21.15.17:0/278934150 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:00:15.698 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.716+0000 7f265159b700 1 --2- 172.21.15.17:0/278934150 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f264c0ffba0 0x7f264c197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:15.699 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.716+0000 7f2650d9a700 1 --2- 172.21.15.17:0/278934150 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f264c100e30 0x7f264c198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:15.699 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.717+0000 7f2651d9c700 1 -- 172.21.15.17:0/278934150 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f264c0ffba0 msgr2=0x7f264c197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:15.699 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.717+0000 7f2651d9c700 1 --2- 172.21.15.17:0/278934150 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f264c0ffba0 0x7f264c197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:15.699 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.717+0000 7f2651d9c700 1 -- 172.21.15.17:0/278934150 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f264c100e30 msgr2=0x7f264c198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:15.699 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.717+0000 7f2651d9c700 1 --2- 172.21.15.17:0/278934150 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f264c100e30 0x7f264c198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:15.699 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.717+0000 7f2651d9c700 1 -- 172.21.15.17:0/278934150 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2644009c40 con 0x7f264c1017e0 2023-07-14T12:00:15.699 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.717+0000 7f2651d9c700 1 --2- 172.21.15.17:0/278934150 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f264c1017e0 0x7f264c19f0f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f264400c370 tx=0x7f264400c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:15.699 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.717+0000 7f26427fc700 1 -- 172.21.15.17:0/278934150 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2644017070 con 0x7f264c1017e0 2023-07-14T12:00:15.700 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.717+0000 7f26427fc700 1 -- 172.21.15.17:0/278934150 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f264400f040 con 0x7f264c1017e0 2023-07-14T12:00:15.700 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.717+0000 7f26537ff700 1 -- 172.21.15.17:0/278934150 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f263c0095f0 con 0x7f264c1017e0 2023-07-14T12:00:15.700 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.717+0000 7f26427fc700 1 -- 172.21.15.17:0/278934150 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f264400e050 con 0x7f264c1017e0 2023-07-14T12:00:15.700 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.717+0000 7f26537ff700 1 -- 172.21.15.17:0/278934150 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f264c19ff90 con 0x7f264c1017e0 2023-07-14T12:00:15.700 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.719+0000 7f26537ff700 1 -- 172.21.15.17:0/278934150 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f264c10b3c0 con 0x7f264c1017e0 2023-07-14T12:00:15.701 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.721+0000 7f26427fc700 1 -- 172.21.15.17:0/278934150 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f264400cb60 con 0x7f264c1017e0 2023-07-14T12:00:15.702 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.723+0000 7f26427fc700 1 --2- 172.21.15.17:0/278934150 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f26380a1150 0x7f26380a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:15.703 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.723+0000 7f26427fc700 1 -- 172.21.15.17:0/278934150 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f26440cc5a0 con 0x7f264c1017e0 2023-07-14T12:00:15.703 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.723+0000 7f265159b700 1 --2- 172.21.15.17:0/278934150 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f26380a1150 0x7f26380a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:15.703 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.724+0000 7f265159b700 1 --2- 172.21.15.17:0/278934150 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f26380a1150 0x7f26380a3600 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2648009c60 tx=0x7f2648009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:15.706 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.727+0000 7f26427fc700 1 -- 172.21.15.17:0/278934150 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f26440959e0 con 0x7f264c1017e0 2023-07-14T12:00:15.966 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.986+0000 7f26537ff700 1 -- 172.21.15.17:0/278934150 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f264c1a05a0 con 0x7f264c1017e0 2023-07-14T12:00:15.966 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.987+0000 7f26427fc700 1 -- 172.21.15.17:0/278934150 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10812 (secure 0 0 0) 0x7f2644099000 con 0x7f264c1017e0 2023-07-14T12:00:15.967 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T12:00:15.967 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/55384193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":55384193},{"type":"v1","addr":"172.21.15.17:6835","nonce":55384193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/424956554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":424956554},{"type":"v1","addr":"172.21.15.17:6837","nonce":424956554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/326319712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":326319712},{"type":"v1","addr":"172.21.15.93:6833","nonce":326319712}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/878913679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":878913679},{"type":"v1","addr":"172.21.15.97:6833","nonce":878913679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/2779420503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":2779420503},{"type":"v1","addr":"172.21.15.93:6835","nonce":2779420503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2740799393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2740799393},{"type":"v1","addr":"172.21.15.97:6835","nonce":2740799393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24509,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/1880673911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":1880673911},{"type":"v1","addr":"172.21.15.93:6837","nonce":1880673911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3684187131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3684187131},{"type":"v1","addr":"172.21.15.97:6837","nonce":3684187131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T11:59:43.292866+0000","modified":"2023-07-14T12:00:15.473993+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3],"up":{"mds_0":24565,"mds_1":24557,"mds_2":24541,"mds_3":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24533":{"gid":24533,"name":"g","rank":3,"incarnation":23,"state":"up:creating","state_seq":1,"addr":"172.21.15.17:6839/2976493015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":2976493015},{"type":"v1","addr":"172.21.15.17:6839","nonce":2976493015}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.97:6839/3267815331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3267815331},{"type":"v1","addr":"172.21.15.97:6839","nonce":3267815331}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.93:6839/1281342428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":1281342428},{"type":"v1","addr":"172.21.15.93:6839","nonce":1281342428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6841/3905201143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6840","nonce":3905201143},{"type":"v1","addr":"172.21.15.17:6841","nonce":3905201143}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T12:00:15.969 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.989+0000 7f26537ff700 1 -- 172.21.15.17:0/278934150 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f26380a1150 msgr2=0x7f26380a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:15.969 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.989+0000 7f26537ff700 1 --2- 172.21.15.17:0/278934150 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f26380a1150 0x7f26380a3600 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2648009c60 tx=0x7f2648009260 comp rx=0 tx=0).stop 2023-07-14T12:00:15.969 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.989+0000 7f26537ff700 1 -- 172.21.15.17:0/278934150 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f264c1017e0 msgr2=0x7f264c19f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:15.969 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.989+0000 7f26537ff700 1 --2- 172.21.15.17:0/278934150 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f264c1017e0 0x7f264c19f0f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f264400c370 tx=0x7f264400c680 comp rx=0 tx=0).stop 2023-07-14T12:00:15.969 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.989+0000 7f26537ff700 1 -- 172.21.15.17:0/278934150 shutdown_connections 2023-07-14T12:00:15.969 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.989+0000 7f26537ff700 1 --2- 172.21.15.17:0/278934150 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f26380a1150 0x7f26380a3600 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:15.970 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.989+0000 7f26537ff700 1 --2- 172.21.15.17:0/278934150 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f264c1017e0 0x7f264c19f0f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:15.970 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.989+0000 7f26537ff700 1 --2- 172.21.15.17:0/278934150 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f264c0ffba0 0x7f264c197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:15.970 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.989+0000 7f26537ff700 1 --2- 172.21.15.17:0/278934150 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f264c100e30 0x7f264c198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:15.970 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.989+0000 7f26537ff700 1 -- 172.21.15.17:0/278934150 >> 172.21.15.17:0/278934150 conn(0x7f264c0fb1c0 msgr2=0x7f264c0fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:15.970 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.989+0000 7f26537ff700 1 -- 172.21.15.17:0/278934150 shutdown_connections 2023-07-14T12:00:15.970 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:15.990+0000 7f26537ff700 1 -- 172.21.15.17:0/278934150 wait complete. 2023-07-14T12:00:15.971 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 23 2023-07-14T12:00:15.988 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T12:00:16.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:15 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/3133303889' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-07-14T12:00:16.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:15 smithi093 ceph-mon[128922]: mds.? [v2:172.21.15.97:6838/3267815331,v1:172.21.15.97:6839/3267815331] up:active 2023-07-14T12:00:16.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:15 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/3133303889' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-07-14T12:00:16.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:15 smithi093 ceph-mon[128922]: daemon mds.g assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-07-14T12:00:16.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:15 smithi093 ceph-mon[128922]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 9 up:standby 2023-07-14T12:00:16.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:15 smithi093 ceph-mon[128922]: fsmap cephfs:4 {0=a=up:active,1=b=up:active,2=i=up:active,3=g=up:creating} 8 up:standby 2023-07-14T12:00:16.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:15 smithi093 ceph-mon[128922]: daemon mds.g is now active in filesystem cephfs as rank 3 2023-07-14T12:00:16.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:15 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/3133303889' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-07-14T12:00:16.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:15 smithi097 ceph-mon[129313]: mds.? [v2:172.21.15.97:6838/3267815331,v1:172.21.15.97:6839/3267815331] up:active 2023-07-14T12:00:16.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:15 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/3133303889' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-07-14T12:00:16.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:15 smithi097 ceph-mon[129313]: daemon mds.g assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-07-14T12:00:16.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:15 smithi097 ceph-mon[129313]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 9 up:standby 2023-07-14T12:00:16.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:15 smithi097 ceph-mon[129313]: fsmap cephfs:4 {0=a=up:active,1=b=up:active,2=i=up:active,3=g=up:creating} 8 up:standby 2023-07-14T12:00:16.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:15 smithi097 ceph-mon[129313]: daemon mds.g is now active in filesystem cephfs as rank 3 2023-07-14T12:00:16.144 INFO:journalctl@ceph.mds.a.smithi017.stdout:Jul 14 12:00:16 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-a[161183]: 2023-07-14T12:00:16.004+0000 7fcd1b40b700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-07-14T12:00:16.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:15 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/3133303889' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-07-14T12:00:16.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:15 smithi017 ceph-mon[126339]: mds.? [v2:172.21.15.97:6838/3267815331,v1:172.21.15.97:6839/3267815331] up:active 2023-07-14T12:00:16.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:15 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/3133303889' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-07-14T12:00:16.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:15 smithi017 ceph-mon[126339]: daemon mds.g assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-07-14T12:00:16.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:15 smithi017 ceph-mon[126339]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=i=up:active} 9 up:standby 2023-07-14T12:00:16.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:15 smithi017 ceph-mon[126339]: fsmap cephfs:4 {0=a=up:active,1=b=up:active,2=i=up:active,3=g=up:creating} 8 up:standby 2023-07-14T12:00:16.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:15 smithi017 ceph-mon[126339]: daemon mds.g is now active in filesystem cephfs as rank 3 2023-07-14T12:00:16.194 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.214+0000 7ff95efff700 1 -- 172.21.15.17:0/1539981681 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff958100e30 msgr2=0x7ff9581012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:16.194 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.214+0000 7ff95efff700 1 --2- 172.21.15.17:0/1539981681 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff958100e30 0x7ff9581012a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff944009990 tx=0x7ff944009ca0 comp rx=0 tx=0).stop 2023-07-14T12:00:16.194 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.214+0000 7ff95efff700 1 -- 172.21.15.17:0/1539981681 shutdown_connections 2023-07-14T12:00:16.194 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.214+0000 7ff95efff700 1 --2- 172.21.15.17:0/1539981681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9581017e0 0x7ff958108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:16.194 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.214+0000 7ff95efff700 1 --2- 172.21.15.17:0/1539981681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9580ffba0 0x7ff9580fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:16.194 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.214+0000 7ff95efff700 1 --2- 172.21.15.17:0/1539981681 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff958100e30 0x7ff9581012a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:16.195 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.214+0000 7ff95efff700 1 -- 172.21.15.17:0/1539981681 >> 172.21.15.17:0/1539981681 conn(0x7ff9580fb1c0 msgr2=0x7ff9580fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:16.195 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.214+0000 7ff95efff700 1 -- 172.21.15.17:0/1539981681 shutdown_connections 2023-07-14T12:00:16.195 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.214+0000 7ff95efff700 1 -- 172.21.15.17:0/1539981681 wait complete. 2023-07-14T12:00:16.196 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.215+0000 7ff95efff700 1 Processor -- start 2023-07-14T12:00:16.197 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.215+0000 7ff95efff700 1 -- start start 2023-07-14T12:00:16.197 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.215+0000 7ff95efff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9580ffba0 0x7ff958197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:16.197 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.216+0000 7ff95efff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff958100e30 0x7ff958198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:16.197 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.216+0000 7ff95efff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff9581017e0 0x7ff95819f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:16.197 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.216+0000 7ff95efff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95819f640 con 0x7ff9580ffba0 2023-07-14T12:00:16.197 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.216+0000 7ff95efff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95819f780 con 0x7ff9581017e0 2023-07-14T12:00:16.197 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.216+0000 7ff95efff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95819f910 con 0x7ff958100e30 2023-07-14T12:00:16.198 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.216+0000 7ff95cd9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9580ffba0 0x7ff958197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:16.198 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.216+0000 7ff95cd9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9580ffba0 0x7ff958197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44936/0 (socket says 172.21.15.17:44936) 2023-07-14T12:00:16.198 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.216+0000 7ff95cd9b700 1 -- 172.21.15.17:0/3153423548 learned_addr learned my addr 172.21.15.17:0/3153423548 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:00:16.198 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.216+0000 7ff957fff700 1 --2- 172.21.15.17:0/3153423548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff958100e30 0x7ff958198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:16.198 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.216+0000 7ff95d59c700 1 --2- 172.21.15.17:0/3153423548 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff9581017e0 0x7ff95819f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:16.198 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.216+0000 7ff95cd9b700 1 -- 172.21.15.17:0/3153423548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff958100e30 msgr2=0x7ff958198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:16.198 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.216+0000 7ff95cd9b700 1 --2- 172.21.15.17:0/3153423548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff958100e30 0x7ff958198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:16.199 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.216+0000 7ff95cd9b700 1 -- 172.21.15.17:0/3153423548 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff9581017e0 msgr2=0x7ff95819f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:16.199 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.216+0000 7ff95cd9b700 1 --2- 172.21.15.17:0/3153423548 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff9581017e0 0x7ff95819f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:16.199 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.216+0000 7ff95cd9b700 1 -- 172.21.15.17:0/3153423548 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff94c009520 con 0x7ff9580ffba0 2023-07-14T12:00:16.199 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.216+0000 7ff95cd9b700 1 --2- 172.21.15.17:0/3153423548 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9580ffba0 0x7ff958197ac0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff94c00eb30 tx=0x7ff94c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:16.199 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.217+0000 7ff955ffb700 1 -- 172.21.15.17:0/3153423548 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94c009880 con 0x7ff9580ffba0 2023-07-14T12:00:16.199 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.217+0000 7ff955ffb700 1 -- 172.21.15.17:0/3153423548 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff94c0044e0 con 0x7ff9580ffba0 2023-07-14T12:00:16.199 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.217+0000 7ff95efff700 1 -- 172.21.15.17:0/3153423548 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9440095f0 con 0x7ff9580ffba0 2023-07-14T12:00:16.199 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.217+0000 7ff955ffb700 1 -- 172.21.15.17:0/3153423548 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94c003d80 con 0x7ff9580ffba0 2023-07-14T12:00:16.200 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.217+0000 7ff95efff700 1 -- 172.21.15.17:0/3153423548 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff95819ff70 con 0x7ff9580ffba0 2023-07-14T12:00:16.200 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.218+0000 7ff95efff700 1 -- 172.21.15.17:0/3153423548 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff95810b370 con 0x7ff9580ffba0 2023-07-14T12:00:16.200 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.220+0000 7ff955ffb700 1 -- 172.21.15.17:0/3153423548 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff94c003f20 con 0x7ff9580ffba0 2023-07-14T12:00:16.201 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.221+0000 7ff955ffb700 1 --2- 172.21.15.17:0/3153423548 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7ff9480a1150 0x7ff9480a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:16.201 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.221+0000 7ff955ffb700 1 -- 172.21.15.17:0/3153423548 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff94c014070 con 0x7ff9580ffba0 2023-07-14T12:00:16.201 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.221+0000 7ff957fff700 1 --2- 172.21.15.17:0/3153423548 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7ff9480a1150 0x7ff9480a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:16.202 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.222+0000 7ff957fff700 1 --2- 172.21.15.17:0/3153423548 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7ff9480a1150 0x7ff9480a3600 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff94400b2e0 tx=0x7ff944005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:16.204 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.225+0000 7ff955ffb700 1 -- 172.21.15.17:0/3153423548 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff94c096590 con 0x7ff9580ffba0 2023-07-14T12:00:16.414 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.435+0000 7ff95efff700 1 -- 172.21.15.17:0/3153423548 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff95810b530 con 0x7ff9580ffba0 2023-07-14T12:00:16.416 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.437+0000 7ff955ffb700 1 -- 172.21.15.17:0/3153423548 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24336 (secure 0 0 0) 0x7ff94c07ff20 con 0x7ff9580ffba0 2023-07-14T12:00:16.416 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T12:00:16.417 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":90,"fsid":"b3c386cc-223c-11ee-9b33-001a4aab830c","created":"2023-07-14T11:51:59.992960+0000","modified":"2023-07-14T12:00:06.800621+0000","last_up_change":"2023-07-14T11:58:55.037769+0000","last_in_change":"2023-07-14T11:58:37.114870+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:55:28.237002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:59:41.560502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T11:59:42.725461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:59:46.267029+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"57003438-a83c-4c8e-a029-56414740895c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6803","nonce":3588366384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6805","nonce":3588366384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6809","nonce":3588366384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6807","nonce":3588366384}]},"public_addr":"172.21.15.17:6803/3588366384","cluster_addr":"172.21.15.17:6805/3588366384","heartbeat_back_addr":"172.21.15.17:6809/3588366384","heartbeat_front_addr":"172.21.15.17:6807/3588366384","state":["exists","up"]},{"osd":1,"uuid":"f56b6337-d8a9-4411-b9d8-b6863dc98556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6801","nonce":3147203064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6803","nonce":3147203064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6807","nonce":3147203064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6805","nonce":3147203064}]},"public_addr":"172.21.15.93:6801/3147203064","cluster_addr":"172.21.15.93:6803/3147203064","heartbeat_back_addr":"172.21.15.93:6807/3147203064","heartbeat_front_addr":"172.21.15.93:6805/3147203064","state":["exists","up"]},{"osd":2,"uuid":"fc34bb7f-551f-46e1-99c0-59924d6c1346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6801","nonce":1877269815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6803","nonce":1877269815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6807","nonce":1877269815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6805","nonce":1877269815}]},"public_addr":"172.21.15.97:6801/1877269815","cluster_addr":"172.21.15.97:6803/1877269815","heartbeat_back_addr":"172.21.15.97:6807/1877269815","heartbeat_front_addr":"172.21.15.97:6805/1877269815","state":["exists","up"]},{"osd":3,"uuid":"37a1583c-4fb8-412a-a0c8-f83b1ff26a91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6811","nonce":2784619821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6813","nonce":2784619821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6817","nonce":2784619821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6815","nonce":2784619821}]},"public_addr":"172.21.15.17:6811/2784619821","cluster_addr":"172.21.15.17:6813/2784619821","heartbeat_back_addr":"172.21.15.17:6817/2784619821","heartbeat_front_addr":"172.21.15.17:6815/2784619821","state":["exists","up"]},{"osd":4,"uuid":"f4b9f377-d519-4b14-ac4a-f1debaeed9f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6809","nonce":3151801173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6811","nonce":3151801173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6815","nonce":3151801173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6813","nonce":3151801173}]},"public_addr":"172.21.15.93:6809/3151801173","cluster_addr":"172.21.15.93:6811/3151801173","heartbeat_back_addr":"172.21.15.93:6815/3151801173","heartbeat_front_addr":"172.21.15.93:6813/3151801173","state":["exists","up"]},{"osd":5,"uuid":"2d2af354-0535-4ef9-b166-3ef219498db8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6809","nonce":2184413924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6811","nonce":2184413924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6815","nonce":2184413924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6813","nonce":2184413924}]},"public_addr":"172.21.15.97:6809/2184413924","cluster_addr":"172.21.15.97:6811/2184413924","heartbeat_back_addr":"172.21.15.97:6815/2184413924","heartbeat_front_addr":"172.21.15.97:6813/2184413924","state":["exists","up"]},{"osd":6,"uuid":"9cdf0c4b-62dc-4168-864e-a9d1650a9ea8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6819","nonce":50486581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6821","nonce":50486581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6825","nonce":50486581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6823","nonce":50486581}]},"public_addr":"172.21.15.17:6819/50486581","cluster_addr":"172.21.15.17:6821/50486581","heartbeat_back_addr":"172.21.15.17:6825/50486581","heartbeat_front_addr":"172.21.15.17:6823/50486581","state":["exists","up"]},{"osd":7,"uuid":"09f60823-5491-45cd-9324-6ae0d56ce262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6817","nonce":2596912392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6819","nonce":2596912392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6823","nonce":2596912392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6821","nonce":2596912392}]},"public_addr":"172.21.15.93:6817/2596912392","cluster_addr":"172.21.15.93:6819/2596912392","heartbeat_back_addr":"172.21.15.93:6823/2596912392","heartbeat_front_addr":"172.21.15.93:6821/2596912392","state":["exists","up"]},{"osd":8,"uuid":"8c5771c9-718e-4313-a2b6-5e1c37c53f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6817","nonce":1965492859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6819","nonce":1965492859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6823","nonce":1965492859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6821","nonce":1965492859}]},"public_addr":"172.21.15.97:6817/1965492859","cluster_addr":"172.21.15.97:6819/1965492859","heartbeat_back_addr":"172.21.15.97:6823/1965492859","heartbeat_front_addr":"172.21.15.97:6821/1965492859","state":["exists","up"]},{"osd":9,"uuid":"1197b8a3-c09f-4c57-aba1-9e988a1620b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6827","nonce":2512645700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6829","nonce":2512645700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6833","nonce":2512645700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6831","nonce":2512645700}]},"public_addr":"172.21.15.17:6827/2512645700","cluster_addr":"172.21.15.17:6829/2512645700","heartbeat_back_addr":"172.21.15.17:6833/2512645700","heartbeat_front_addr":"172.21.15.17:6831/2512645700","state":["exists","up"]},{"osd":10,"uuid":"f99918c2-cdd8-4a7a-be2e-d1d6e0f828a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6825","nonce":264429558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6827","nonce":264429558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6831","nonce":264429558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6829","nonce":264429558}]},"public_addr":"172.21.15.93:6825/264429558","cluster_addr":"172.21.15.93:6827/264429558","heartbeat_back_addr":"172.21.15.93:6831/264429558","heartbeat_front_addr":"172.21.15.93:6829/264429558","state":["exists","up"]},{"osd":11,"uuid":"6bbb5b79-8d4d-441f-9583-203df6c78cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6825","nonce":3389913754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6827","nonce":3389913754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6831","nonce":3389913754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6829","nonce":3389913754}]},"public_addr":"172.21.15.97:6825/3389913754","cluster_addr":"172.21.15.97:6827/3389913754","heartbeat_back_addr":"172.21.15.97:6831/3389913754","heartbeat_front_addr":"172.21.15.97:6829/3389913754","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:54:41.079618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:03.207899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:24.136961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:50.392609+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:56:12.136237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:56:33.212314+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:00.243646+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:22.059234+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:43.232399+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:10.043896+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:31.910780+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:53.158983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.9","mappings":[{"from":5,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/2084033276":"2023-07-15T11:53:05.639597+0000","172.21.15.17:6801/1059284716":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/3651785636":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/1373538024":"2023-07-15T11:52:37.887946+0000","172.21.15.17:0/3560395020":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6800/1059284716":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/4262912861":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/2258375202":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6800/2804227742":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6801/2804227742":"2023-07-15T11:52:37.887946+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T12:00:16.418 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.439+0000 7ff95efff700 1 -- 172.21.15.17:0/3153423548 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7ff9480a1150 msgr2=0x7ff9480a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:16.419 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.439+0000 7ff95efff700 1 --2- 172.21.15.17:0/3153423548 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7ff9480a1150 0x7ff9480a3600 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff94400b2e0 tx=0x7ff944005fb0 comp rx=0 tx=0).stop 2023-07-14T12:00:16.419 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.439+0000 7ff95efff700 1 -- 172.21.15.17:0/3153423548 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9580ffba0 msgr2=0x7ff958197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:16.419 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.439+0000 7ff95efff700 1 --2- 172.21.15.17:0/3153423548 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9580ffba0 0x7ff958197ac0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff94c00eb30 tx=0x7ff94c00ee40 comp rx=0 tx=0).stop 2023-07-14T12:00:16.419 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.439+0000 7ff95efff700 1 -- 172.21.15.17:0/3153423548 shutdown_connections 2023-07-14T12:00:16.419 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.439+0000 7ff95efff700 1 --2- 172.21.15.17:0/3153423548 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7ff9480a1150 0x7ff9480a3600 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:16.419 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.439+0000 7ff95efff700 1 --2- 172.21.15.17:0/3153423548 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff9581017e0 0x7ff95819f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:16.419 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.439+0000 7ff95efff700 1 --2- 172.21.15.17:0/3153423548 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9580ffba0 0x7ff958197ac0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:16.420 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.439+0000 7ff95efff700 1 --2- 172.21.15.17:0/3153423548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff958100e30 0x7ff958198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:16.420 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.439+0000 7ff95efff700 1 -- 172.21.15.17:0/3153423548 >> 172.21.15.17:0/3153423548 conn(0x7ff9580fb1c0 msgr2=0x7ff9580fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:16.420 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.439+0000 7ff95efff700 1 -- 172.21.15.17:0/3153423548 shutdown_connections 2023-07-14T12:00:16.420 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.439+0000 7ff95efff700 1 -- 172.21.15.17:0/3153423548 wait complete. 2023-07-14T12:00:16.437 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-14T12:00:16.438 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-14T12:00:16.647 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.668+0000 7f093adff700 1 -- 172.21.15.17:0/1911356140 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0934101de0 msgr2=0x7f09341021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:16.648 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.668+0000 7f093adff700 1 --2- 172.21.15.17:0/1911356140 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0934101de0 0x7f09341021f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0928009990 tx=0x7f0928009ca0 comp rx=0 tx=0).stop 2023-07-14T12:00:16.648 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.668+0000 7f093adff700 1 -- 172.21.15.17:0/1911356140 shutdown_connections 2023-07-14T12:00:16.648 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.668+0000 7f093adff700 1 --2- 172.21.15.17:0/1911356140 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0934103960 0x7f093410a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:16.648 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.668+0000 7f093adff700 1 --2- 172.21.15.17:0/1911356140 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0934101de0 0x7f09341021f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:16.649 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.668+0000 7f093adff700 1 --2- 172.21.15.17:0/1911356140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0934102fd0 0x7f0934103420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:16.649 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.668+0000 7f093adff700 1 -- 172.21.15.17:0/1911356140 >> 172.21.15.17:0/1911356140 conn(0x7f09340fd400 msgr2=0x7f09340ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:16.649 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.668+0000 7f093adff700 1 -- 172.21.15.17:0/1911356140 shutdown_connections 2023-07-14T12:00:16.649 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.668+0000 7f093adff700 1 -- 172.21.15.17:0/1911356140 wait complete. 2023-07-14T12:00:16.651 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.669+0000 7f093adff700 1 Processor -- start 2023-07-14T12:00:16.651 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.669+0000 7f093adff700 1 -- start start 2023-07-14T12:00:16.651 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.670+0000 7f093adff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0934101de0 0x7f0934197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:16.651 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.670+0000 7f093adff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0934102fd0 0x7f0934197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:16.651 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.670+0000 7f093adff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0934103960 0x7f093419f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:16.651 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.670+0000 7f093adff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093419f620 con 0x7f0934101de0 2023-07-14T12:00:16.651 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.670+0000 7f093adff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093419f760 con 0x7f0934102fd0 2023-07-14T12:00:16.652 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.670+0000 7f093adff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093419f8f0 con 0x7f0934103960 2023-07-14T12:00:16.652 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.670+0000 7f0938b9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0934101de0 0x7f0934197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:16.652 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.670+0000 7f0938b9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0934101de0 0x7f0934197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44954/0 (socket says 172.21.15.17:44954) 2023-07-14T12:00:16.652 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.670+0000 7f0938b9b700 1 -- 172.21.15.17:0/3184598835 learned_addr learned my addr 172.21.15.17:0/3184598835 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:00:16.652 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.670+0000 7f093939c700 1 --2- 172.21.15.17:0/3184598835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0934103960 0x7f093419f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:16.652 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.670+0000 7f0933fff700 1 --2- 172.21.15.17:0/3184598835 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0934102fd0 0x7f0934197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:16.652 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.670+0000 7f0938b9b700 1 -- 172.21.15.17:0/3184598835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0934103960 msgr2=0x7f093419f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:16.653 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.670+0000 7f0938b9b700 1 --2- 172.21.15.17:0/3184598835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0934103960 0x7f093419f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:16.653 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.670+0000 7f0938b9b700 1 -- 172.21.15.17:0/3184598835 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0934102fd0 msgr2=0x7f0934197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:16.653 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.670+0000 7f0938b9b700 1 --2- 172.21.15.17:0/3184598835 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0934102fd0 0x7f0934197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:16.653 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.670+0000 7f0938b9b700 1 -- 172.21.15.17:0/3184598835 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09280095f0 con 0x7f0934101de0 2023-07-14T12:00:16.653 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.670+0000 7f0933fff700 1 --2- 172.21.15.17:0/3184598835 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0934102fd0 0x7f0934197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T12:00:16.653 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.670+0000 7f0938b9b700 1 --2- 172.21.15.17:0/3184598835 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0934101de0 0x7f0934197aa0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0928000c00 tx=0x7f09280047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:16.653 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.671+0000 7f0931ffb700 1 -- 172.21.15.17:0/3184598835 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092801d070 con 0x7f0934101de0 2023-07-14T12:00:16.654 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.671+0000 7f093adff700 1 -- 172.21.15.17:0/3184598835 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f093419fb70 con 0x7f0934101de0 2023-07-14T12:00:16.654 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.671+0000 7f0931ffb700 1 -- 172.21.15.17:0/3184598835 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0928022470 con 0x7f0934101de0 2023-07-14T12:00:16.654 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.671+0000 7f0931ffb700 1 -- 172.21.15.17:0/3184598835 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0928017650 con 0x7f0934101de0 2023-07-14T12:00:16.654 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.671+0000 7f093adff700 1 -- 172.21.15.17:0/3184598835 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f093419ffa0 con 0x7f0934101de0 2023-07-14T12:00:16.654 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.672+0000 7f093adff700 1 -- 172.21.15.17:0/3184598835 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0934109330 con 0x7f0934101de0 2023-07-14T12:00:16.655 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.674+0000 7f0931ffb700 1 -- 172.21.15.17:0/3184598835 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f09280225e0 con 0x7f0934101de0 2023-07-14T12:00:16.655 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.676+0000 7f0931ffb700 1 --2- 172.21.15.17:0/3184598835 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f09240a1150 0x7f09240a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:16.655 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.676+0000 7f0931ffb700 1 -- 172.21.15.17:0/3184598835 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f09280ce290 con 0x7f0934101de0 2023-07-14T12:00:16.655 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.676+0000 7f0933fff700 1 --2- 172.21.15.17:0/3184598835 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f09240a1150 0x7f09240a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:16.656 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.677+0000 7f0933fff700 1 --2- 172.21.15.17:0/3184598835 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f09240a1150 0x7f09240a3600 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0920009cb0 tx=0x7f0920009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:16.658 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.679+0000 7f0931ffb700 1 -- 172.21.15.17:0/3184598835 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0928097700 con 0x7f0934101de0 2023-07-14T12:00:16.864 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.885+0000 7f093adff700 1 -- 172.21.15.17:0/3184598835 --> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0934060950 con 0x7f09240a1150 2023-07-14T12:00:16.880 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.900+0000 7f0931ffb700 1 -- 172.21.15.17:0/3184598835 <== mgr.14152 v2:172.21.15.17:6800/4237449858 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+423345 (secure 0 0 0) 0x7f0934060950 con 0x7f09240a1150 2023-07-14T12:00:16.888 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T12:00:16.892 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.905+0000 7f093adff700 1 -- 172.21.15.17:0/3184598835 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f09240a1150 msgr2=0x7f09240a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:16.893 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.905+0000 7f093adff700 1 --2- 172.21.15.17:0/3184598835 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f09240a1150 0x7f09240a3600 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0920009cb0 tx=0x7f0920009260 comp rx=0 tx=0).stop 2023-07-14T12:00:16.893 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.905+0000 7f093adff700 1 -- 172.21.15.17:0/3184598835 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0934101de0 msgr2=0x7f0934197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:16.893 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.905+0000 7f093adff700 1 --2- 172.21.15.17:0/3184598835 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0934101de0 0x7f0934197aa0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0928000c00 tx=0x7f09280047a0 comp rx=0 tx=0).stop 2023-07-14T12:00:16.894 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.905+0000 7f093adff700 1 -- 172.21.15.17:0/3184598835 shutdown_connections 2023-07-14T12:00:16.894 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.905+0000 7f093adff700 1 --2- 172.21.15.17:0/3184598835 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f09240a1150 0x7f09240a3600 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:16.894 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.905+0000 7f093adff700 1 --2- 172.21.15.17:0/3184598835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0934103960 0x7f093419f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:16.894 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.905+0000 7f093adff700 1 --2- 172.21.15.17:0/3184598835 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0934101de0 0x7f0934197aa0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:16.894 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.905+0000 7f093adff700 1 --2- 172.21.15.17:0/3184598835 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0934102fd0 0x7f0934197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:16.894 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.905+0000 7f093adff700 1 -- 172.21.15.17:0/3184598835 >> 172.21.15.17:0/3184598835 conn(0x7f09340fd400 msgr2=0x7f0934106470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:16.894 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.905+0000 7f093adff700 1 -- 172.21.15.17:0/3184598835 shutdown_connections 2023-07-14T12:00:16.895 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:16.905+0000 7f093adff700 1 -- 172.21.15.17:0/3184598835 wait complete. 2023-07-14T12:00:16.895 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-07-14T12:00:16.904 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2023-07-14T12:00:15.732612+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596780,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":255,"num_write_kb":2969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335052,"kb_used_data":11628,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606772,"statfs":{"total":1151940427776,"available":1151597334528,"internally_reserved":0,"allocated":11907072,"data_stored":7620287,"data_compressed":319254,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1961,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.931592"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.813006+0000","last_change":"2023-07-14T11:59:43.468306+0000","last_active":"2023-07-14T12:00:06.813006+0000","last_peered":"2023-07-14T12:00:06.813006+0000","last_clean":"2023-07-14T12:00:06.813006+0000","last_became_active":"2023-07-14T11:59:43.467506+0000","last_became_peered":"2023-07-14T11:59:43.467506+0000","last_unstale":"2023-07-14T12:00:06.813006+0000","last_undegraded":"2023-07-14T12:00:06.813006+0000","last_fullsized":"2023-07-14T12:00:06.813006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:49:26.343044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.814808+0000","last_change":"2023-07-14T11:59:43.462562+0000","last_active":"2023-07-14T12:00:06.814808+0000","last_peered":"2023-07-14T12:00:06.814808+0000","last_clean":"2023-07-14T12:00:06.814808+0000","last_became_active":"2023-07-14T11:59:43.462126+0000","last_became_peered":"2023-07-14T11:59:43.462126+0000","last_unstale":"2023-07-14T12:00:06.814808+0000","last_undegraded":"2023-07-14T12:00:06.814808+0000","last_fullsized":"2023-07-14T12:00:06.814808+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:20:10.445422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.813957+0000","last_change":"2023-07-14T11:59:43.470205+0000","last_active":"2023-07-14T12:00:06.813957+0000","last_peered":"2023-07-14T12:00:06.813957+0000","last_clean":"2023-07-14T12:00:06.813957+0000","last_became_active":"2023-07-14T11:59:43.469804+0000","last_became_peered":"2023-07-14T11:59:43.469804+0000","last_unstale":"2023-07-14T12:00:06.813957+0000","last_undegraded":"2023-07-14T12:00:06.813957+0000","last_fullsized":"2023-07-14T12:00:06.813957+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:06:21.532674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.807812+0000","last_change":"2023-07-14T11:59:43.464834+0000","last_active":"2023-07-14T12:00:06.807812+0000","last_peered":"2023-07-14T12:00:06.807812+0000","last_clean":"2023-07-14T12:00:06.807812+0000","last_became_active":"2023-07-14T11:59:43.464252+0000","last_became_peered":"2023-07-14T11:59:43.464252+0000","last_unstale":"2023-07-14T12:00:06.807812+0000","last_undegraded":"2023-07-14T12:00:06.807812+0000","last_fullsized":"2023-07-14T12:00:06.807812+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:30:26.528049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808013+0000","last_change":"2023-07-14T11:59:43.460468+0000","last_active":"2023-07-14T12:00:06.808013+0000","last_peered":"2023-07-14T12:00:06.808013+0000","last_clean":"2023-07-14T12:00:06.808013+0000","last_became_active":"2023-07-14T11:59:43.460050+0000","last_became_peered":"2023-07-14T11:59:43.460050+0000","last_unstale":"2023-07-14T12:00:06.808013+0000","last_undegraded":"2023-07-14T12:00:06.808013+0000","last_fullsized":"2023-07-14T12:00:06.808013+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:43:21.915056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808330+0000","last_change":"2023-07-14T11:59:43.458605+0000","last_active":"2023-07-14T12:00:06.808330+0000","last_peered":"2023-07-14T12:00:06.808330+0000","last_clean":"2023-07-14T12:00:06.808330+0000","last_became_active":"2023-07-14T11:59:43.458233+0000","last_became_peered":"2023-07-14T11:59:43.458233+0000","last_unstale":"2023-07-14T12:00:06.808330+0000","last_undegraded":"2023-07-14T12:00:06.808330+0000","last_fullsized":"2023-07-14T12:00:06.808330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:27:55.968254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.328508+0000","last_change":"2023-07-14T11:59:43.465950+0000","last_active":"2023-07-14T12:00:07.328508+0000","last_peered":"2023-07-14T12:00:07.328508+0000","last_clean":"2023-07-14T12:00:07.328508+0000","last_became_active":"2023-07-14T11:59:43.465652+0000","last_became_peered":"2023-07-14T11:59:43.465652+0000","last_unstale":"2023-07-14T12:00:07.328508+0000","last_undegraded":"2023-07-14T12:00:07.328508+0000","last_fullsized":"2023-07-14T12:00:07.328508+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:48:42.466464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.816094+0000","last_change":"2023-07-14T12:00:06.812609+0000","last_active":"2023-07-14T12:00:06.816094+0000","last_peered":"2023-07-14T12:00:06.816094+0000","last_clean":"2023-07-14T12:00:06.816094+0000","last_became_active":"2023-07-14T12:00:06.812112+0000","last_became_peered":"2023-07-14T12:00:06.812112+0000","last_unstale":"2023-07-14T12:00:06.816094+0000","last_undegraded":"2023-07-14T12:00:06.816094+0000","last_fullsized":"2023-07-14T12:00:06.816094+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:41:31.808111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805908+0000","last_change":"2023-07-14T11:59:43.466351+0000","last_active":"2023-07-14T12:00:06.805908+0000","last_peered":"2023-07-14T12:00:06.805908+0000","last_clean":"2023-07-14T12:00:06.805908+0000","last_became_active":"2023-07-14T11:59:43.465442+0000","last_became_peered":"2023-07-14T11:59:43.465442+0000","last_unstale":"2023-07-14T12:00:06.805908+0000","last_undegraded":"2023-07-14T12:00:06.805908+0000","last_fullsized":"2023-07-14T12:00:06.805908+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:52:04.297945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.809112+0000","last_change":"2023-07-14T11:59:43.457684+0000","last_active":"2023-07-14T12:00:06.809112+0000","last_peered":"2023-07-14T12:00:06.809112+0000","last_clean":"2023-07-14T12:00:06.809112+0000","last_became_active":"2023-07-14T11:59:43.457442+0000","last_became_peered":"2023-07-14T11:59:43.457442+0000","last_unstale":"2023-07-14T12:00:06.809112+0000","last_undegraded":"2023-07-14T12:00:06.809112+0000","last_fullsized":"2023-07-14T12:00:06.809112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:06:41.592243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.328566+0000","last_change":"2023-07-14T11:59:43.463652+0000","last_active":"2023-07-14T12:00:07.328566+0000","last_peered":"2023-07-14T12:00:07.328566+0000","last_clean":"2023-07-14T12:00:07.328566+0000","last_became_active":"2023-07-14T11:59:43.463311+0000","last_became_peered":"2023-07-14T11:59:43.463311+0000","last_unstale":"2023-07-14T12:00:07.328566+0000","last_undegraded":"2023-07-14T12:00:07.328566+0000","last_fullsized":"2023-07-14T12:00:07.328566+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:28:35.177490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808344+0000","last_change":"2023-07-14T11:59:43.464633+0000","last_active":"2023-07-14T12:00:06.808344+0000","last_peered":"2023-07-14T12:00:06.808344+0000","last_clean":"2023-07-14T12:00:06.808344+0000","last_became_active":"2023-07-14T11:59:43.463847+0000","last_became_peered":"2023-07-14T11:59:43.463847+0000","last_unstale":"2023-07-14T12:00:06.808344+0000","last_undegraded":"2023-07-14T12:00:06.808344+0000","last_fullsized":"2023-07-14T12:00:06.808344+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:08:28.236596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808316+0000","last_change":"2023-07-14T11:59:43.465650+0000","last_active":"2023-07-14T12:00:06.808316+0000","last_peered":"2023-07-14T12:00:06.808316+0000","last_clean":"2023-07-14T12:00:06.808316+0000","last_became_active":"2023-07-14T11:59:43.465276+0000","last_became_peered":"2023-07-14T11:59:43.465276+0000","last_unstale":"2023-07-14T12:00:06.808316+0000","last_undegraded":"2023-07-14T12:00:06.808316+0000","last_fullsized":"2023-07-14T12:00:06.808316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:00:25.094505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.812937+0000","last_change":"2023-07-14T11:59:43.466437+0000","last_active":"2023-07-14T12:00:06.812937+0000","last_peered":"2023-07-14T12:00:06.812937+0000","last_clean":"2023-07-14T12:00:06.812937+0000","last_became_active":"2023-07-14T11:59:43.465361+0000","last_became_peered":"2023-07-14T11:59:43.465361+0000","last_unstale":"2023-07-14T12:00:06.812937+0000","last_undegraded":"2023-07-14T12:00:06.812937+0000","last_fullsized":"2023-07-14T12:00:06.812937+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:31:26.002175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.814761+0000","last_change":"2023-07-14T11:59:43.469452+0000","last_active":"2023-07-14T12:00:06.814761+0000","last_peered":"2023-07-14T12:00:06.814761+0000","last_clean":"2023-07-14T12:00:06.814761+0000","last_became_active":"2023-07-14T11:59:43.469112+0000","last_became_peered":"2023-07-14T11:59:43.469112+0000","last_unstale":"2023-07-14T12:00:06.814761+0000","last_undegraded":"2023-07-14T12:00:06.814761+0000","last_fullsized":"2023-07-14T12:00:06.814761+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:25:31.596988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.804882+0000","last_change":"2023-07-14T11:59:43.466746+0000","last_active":"2023-07-14T12:00:06.804882+0000","last_peered":"2023-07-14T12:00:06.804882+0000","last_clean":"2023-07-14T12:00:06.804882+0000","last_became_active":"2023-07-14T11:59:43.466530+0000","last_became_peered":"2023-07-14T11:59:43.466530+0000","last_unstale":"2023-07-14T12:00:06.804882+0000","last_undegraded":"2023-07-14T12:00:06.804882+0000","last_fullsized":"2023-07-14T12:00:06.804882+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:47:15.633490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.812795+0000","last_change":"2023-07-14T11:59:43.468510+0000","last_active":"2023-07-14T12:00:06.812795+0000","last_peered":"2023-07-14T12:00:06.812795+0000","last_clean":"2023-07-14T12:00:06.812795+0000","last_became_active":"2023-07-14T11:59:43.467990+0000","last_became_peered":"2023-07-14T11:59:43.467990+0000","last_unstale":"2023-07-14T12:00:06.812795+0000","last_undegraded":"2023-07-14T12:00:06.812795+0000","last_fullsized":"2023-07-14T12:00:06.812795+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:35:21.496112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805431+0000","last_change":"2023-07-14T11:59:43.465290+0000","last_active":"2023-07-14T12:00:06.805431+0000","last_peered":"2023-07-14T12:00:06.805431+0000","last_clean":"2023-07-14T12:00:06.805431+0000","last_became_active":"2023-07-14T11:59:43.464982+0000","last_became_peered":"2023-07-14T11:59:43.464982+0000","last_unstale":"2023-07-14T12:00:06.805431+0000","last_undegraded":"2023-07-14T12:00:06.805431+0000","last_fullsized":"2023-07-14T12:00:06.805431+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:49:36.157904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.806693+0000","last_change":"2023-07-14T11:59:43.465053+0000","last_active":"2023-07-14T12:00:06.806693+0000","last_peered":"2023-07-14T12:00:06.806693+0000","last_clean":"2023-07-14T12:00:06.806693+0000","last_became_active":"2023-07-14T11:59:43.464759+0000","last_became_peered":"2023-07-14T11:59:43.464759+0000","last_unstale":"2023-07-14T12:00:06.806693+0000","last_undegraded":"2023-07-14T12:00:06.806693+0000","last_fullsized":"2023-07-14T12:00:06.806693+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:41:15.313602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805157+0000","last_change":"2023-07-14T11:59:47.792164+0000","last_active":"2023-07-14T12:00:06.805157+0000","last_peered":"2023-07-14T12:00:06.805157+0000","last_clean":"2023-07-14T12:00:06.805157+0000","last_became_active":"2023-07-14T11:59:47.791749+0000","last_became_peered":"2023-07-14T11:59:47.791749+0000","last_unstale":"2023-07-14T12:00:06.805157+0000","last_undegraded":"2023-07-14T12:00:06.805157+0000","last_fullsized":"2023-07-14T12:00:06.805157+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:32:07.262881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805207+0000","last_change":"2023-07-14T11:59:42.759256+0000","last_active":"2023-07-14T12:00:06.805207+0000","last_peered":"2023-07-14T12:00:06.805207+0000","last_clean":"2023-07-14T12:00:06.805207+0000","last_became_active":"2023-07-14T11:59:42.758938+0000","last_became_peered":"2023-07-14T11:59:42.758938+0000","last_unstale":"2023-07-14T12:00:06.805207+0000","last_undegraded":"2023-07-14T12:00:06.805207+0000","last_fullsized":"2023-07-14T12:00:06.805207+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:21:38.079963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.811011+0000","last_change":"2023-07-14T11:59:43.467503+0000","last_active":"2023-07-14T12:00:06.811011+0000","last_peered":"2023-07-14T12:00:06.811011+0000","last_clean":"2023-07-14T12:00:06.811011+0000","last_became_active":"2023-07-14T11:59:43.467130+0000","last_became_peered":"2023-07-14T11:59:43.467130+0000","last_unstale":"2023-07-14T12:00:06.811011+0000","last_undegraded":"2023-07-14T12:00:06.811011+0000","last_fullsized":"2023-07-14T12:00:06.811011+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:26:08.037607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.807340+0000","last_change":"2023-07-14T11:59:47.783514+0000","last_active":"2023-07-14T12:00:06.807340+0000","last_peered":"2023-07-14T12:00:06.807340+0000","last_clean":"2023-07-14T12:00:06.807340+0000","last_became_active":"2023-07-14T11:59:47.783115+0000","last_became_peered":"2023-07-14T11:59:47.783115+0000","last_unstale":"2023-07-14T12:00:06.807340+0000","last_undegraded":"2023-07-14T12:00:06.807340+0000","last_fullsized":"2023-07-14T12:00:06.807340+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:19:30.538776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"83'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.811088+0000","last_change":"2023-07-14T11:59:42.768975+0000","last_active":"2023-07-14T12:00:06.811088+0000","last_peered":"2023-07-14T12:00:06.811088+0000","last_clean":"2023-07-14T12:00:06.811088+0000","last_became_active":"2023-07-14T11:59:42.768603+0000","last_became_peered":"2023-07-14T11:59:42.768603+0000","last_unstale":"2023-07-14T12:00:06.811088+0000","last_undegraded":"2023-07-14T12:00:06.811088+0000","last_fullsized":"2023-07-14T12:00:06.811088+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:20:32.308882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.813232+0000","last_change":"2023-07-14T11:59:43.474252+0000","last_active":"2023-07-14T12:00:06.813232+0000","last_peered":"2023-07-14T12:00:06.813232+0000","last_clean":"2023-07-14T12:00:06.813232+0000","last_became_active":"2023-07-14T11:59:43.473955+0000","last_became_peered":"2023-07-14T11:59:43.473955+0000","last_unstale":"2023-07-14T12:00:06.813232+0000","last_undegraded":"2023-07-14T12:00:06.813232+0000","last_fullsized":"2023-07-14T12:00:06.813232+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:52:46.217372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.813893+0000","last_change":"2023-07-14T12:00:06.813893+0000","last_active":"2023-07-14T12:00:06.813893+0000","last_peered":"2023-07-14T12:00:06.813893+0000","last_clean":"2023-07-14T12:00:06.813893+0000","last_became_active":"2023-07-14T12:00:06.813568+0000","last_became_peered":"2023-07-14T12:00:06.813568+0000","last_unstale":"2023-07-14T12:00:06.813893+0000","last_undegraded":"2023-07-14T12:00:06.813893+0000","last_fullsized":"2023-07-14T12:00:06.813893+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:57:58.591410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808860+0000","last_change":"2023-07-14T11:59:42.760791+0000","last_active":"2023-07-14T12:00:06.808860+0000","last_peered":"2023-07-14T12:00:06.808860+0000","last_clean":"2023-07-14T12:00:06.808860+0000","last_became_active":"2023-07-14T11:59:42.760422+0000","last_became_peered":"2023-07-14T11:59:42.760422+0000","last_unstale":"2023-07-14T12:00:06.808860+0000","last_undegraded":"2023-07-14T12:00:06.808860+0000","last_fullsized":"2023-07-14T12:00:06.808860+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:10:21.008350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805235+0000","last_change":"2023-07-14T11:59:43.455829+0000","last_active":"2023-07-14T12:00:06.805235+0000","last_peered":"2023-07-14T12:00:06.805235+0000","last_clean":"2023-07-14T12:00:06.805235+0000","last_became_active":"2023-07-14T11:59:43.455541+0000","last_became_peered":"2023-07-14T11:59:43.455541+0000","last_unstale":"2023-07-14T12:00:06.805235+0000","last_undegraded":"2023-07-14T12:00:06.805235+0000","last_fullsized":"2023-07-14T12:00:06.805235+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:42:43.607658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.807537+0000","last_change":"2023-07-14T11:59:47.783496+0000","last_active":"2023-07-14T12:00:06.807537+0000","last_peered":"2023-07-14T12:00:06.807537+0000","last_clean":"2023-07-14T12:00:06.807537+0000","last_became_active":"2023-07-14T11:59:47.783128+0000","last_became_peered":"2023-07-14T11:59:47.783128+0000","last_unstale":"2023-07-14T12:00:06.807537+0000","last_undegraded":"2023-07-14T12:00:06.807537+0000","last_fullsized":"2023-07-14T12:00:06.807537+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:55:13.905682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.807417+0000","last_change":"2023-07-14T11:59:42.757255+0000","last_active":"2023-07-14T12:00:06.807417+0000","last_peered":"2023-07-14T12:00:06.807417+0000","last_clean":"2023-07-14T12:00:06.807417+0000","last_became_active":"2023-07-14T11:59:42.756882+0000","last_became_peered":"2023-07-14T11:59:42.756882+0000","last_unstale":"2023-07-14T12:00:06.807417+0000","last_undegraded":"2023-07-14T12:00:06.807417+0000","last_fullsized":"2023-07-14T12:00:06.807417+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:22:50.954980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808779+0000","last_change":"2023-07-14T11:59:43.465501+0000","last_active":"2023-07-14T12:00:06.808779+0000","last_peered":"2023-07-14T12:00:06.808779+0000","last_clean":"2023-07-14T12:00:06.808779+0000","last_became_active":"2023-07-14T11:59:43.465218+0000","last_became_peered":"2023-07-14T11:59:43.465218+0000","last_unstale":"2023-07-14T12:00:06.808779+0000","last_undegraded":"2023-07-14T12:00:06.808779+0000","last_fullsized":"2023-07-14T12:00:06.808779+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:02:45.983300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.806670+0000","last_change":"2023-07-14T11:59:47.777816+0000","last_active":"2023-07-14T12:00:06.806670+0000","last_peered":"2023-07-14T12:00:06.806670+0000","last_clean":"2023-07-14T12:00:06.806670+0000","last_became_active":"2023-07-14T11:59:47.777373+0000","last_became_peered":"2023-07-14T11:59:47.777373+0000","last_unstale":"2023-07-14T12:00:06.806670+0000","last_undegraded":"2023-07-14T12:00:06.806670+0000","last_fullsized":"2023-07-14T12:00:06.806670+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:58:17.252326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:13.466245+0000","last_change":"2023-07-14T11:59:42.756362+0000","last_active":"2023-07-14T12:00:13.466245+0000","last_peered":"2023-07-14T12:00:13.466245+0000","last_clean":"2023-07-14T12:00:13.466245+0000","last_became_active":"2023-07-14T11:59:42.756074+0000","last_became_peered":"2023-07-14T11:59:42.756074+0000","last_unstale":"2023-07-14T12:00:13.466245+0000","last_undegraded":"2023-07-14T12:00:13.466245+0000","last_fullsized":"2023-07-14T12:00:13.466245+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:22:56.214797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.812341+0000","last_change":"2023-07-14T12:00:06.812341+0000","last_active":"2023-07-14T12:00:06.812341+0000","last_peered":"2023-07-14T12:00:06.812341+0000","last_clean":"2023-07-14T12:00:06.812341+0000","last_became_active":"2023-07-14T12:00:06.811891+0000","last_became_peered":"2023-07-14T12:00:06.811891+0000","last_unstale":"2023-07-14T12:00:06.812341+0000","last_undegraded":"2023-07-14T12:00:06.812341+0000","last_fullsized":"2023-07-14T12:00:06.812341+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:19:54.770200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.806081+0000","last_change":"2023-07-14T11:59:47.785323+0000","last_active":"2023-07-14T12:00:06.806081+0000","last_peered":"2023-07-14T12:00:06.806081+0000","last_clean":"2023-07-14T12:00:06.806081+0000","last_became_active":"2023-07-14T11:59:47.784892+0000","last_became_peered":"2023-07-14T11:59:47.784892+0000","last_unstale":"2023-07-14T12:00:06.806081+0000","last_undegraded":"2023-07-14T12:00:06.806081+0000","last_fullsized":"2023-07-14T12:00:06.806081+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:48:54.236864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"83'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808194+0000","last_change":"2023-07-14T11:59:42.756364+0000","last_active":"2023-07-14T12:00:06.808194+0000","last_peered":"2023-07-14T12:00:06.808194+0000","last_clean":"2023-07-14T12:00:06.808194+0000","last_became_active":"2023-07-14T11:59:42.756080+0000","last_became_peered":"2023-07-14T11:59:42.756080+0000","last_unstale":"2023-07-14T12:00:06.808194+0000","last_undegraded":"2023-07-14T12:00:06.808194+0000","last_fullsized":"2023-07-14T12:00:06.808194+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:55:28.974638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808819+0000","last_change":"2023-07-14T11:59:43.470331+0000","last_active":"2023-07-14T12:00:06.808819+0000","last_peered":"2023-07-14T12:00:06.808819+0000","last_clean":"2023-07-14T12:00:06.808819+0000","last_became_active":"2023-07-14T11:59:43.469806+0000","last_became_peered":"2023-07-14T11:59:43.469806+0000","last_unstale":"2023-07-14T12:00:06.808819+0000","last_undegraded":"2023-07-14T12:00:06.808819+0000","last_fullsized":"2023-07-14T12:00:06.808819+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:31:39.648120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.812748+0000","last_change":"2023-07-14T11:59:47.784083+0000","last_active":"2023-07-14T12:00:06.812748+0000","last_peered":"2023-07-14T12:00:06.812748+0000","last_clean":"2023-07-14T12:00:06.812748+0000","last_became_active":"2023-07-14T11:59:47.783624+0000","last_became_peered":"2023-07-14T11:59:47.783624+0000","last_unstale":"2023-07-14T12:00:06.812748+0000","last_undegraded":"2023-07-14T12:00:06.812748+0000","last_fullsized":"2023-07-14T12:00:06.812748+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:19:59.493479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.809448+0000","last_change":"2023-07-14T11:59:42.764869+0000","last_active":"2023-07-14T12:00:06.809448+0000","last_peered":"2023-07-14T12:00:06.809448+0000","last_clean":"2023-07-14T12:00:06.809448+0000","last_became_active":"2023-07-14T11:59:42.764514+0000","last_became_peered":"2023-07-14T11:59:42.764514+0000","last_unstale":"2023-07-14T12:00:06.809448+0000","last_undegraded":"2023-07-14T12:00:06.809448+0000","last_fullsized":"2023-07-14T12:00:06.809448+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:14:29.041985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.809908+0000","last_change":"2023-07-14T11:59:43.458165+0000","last_active":"2023-07-14T12:00:06.809908+0000","last_peered":"2023-07-14T12:00:06.809908+0000","last_clean":"2023-07-14T12:00:06.809908+0000","last_became_active":"2023-07-14T11:59:43.457829+0000","last_became_peered":"2023-07-14T11:59:43.457829+0000","last_unstale":"2023-07-14T12:00:06.809908+0000","last_undegraded":"2023-07-14T12:00:06.809908+0000","last_fullsized":"2023-07-14T12:00:06.809908+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:43:38.283487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.814633+0000","last_change":"2023-07-14T11:59:47.782676+0000","last_active":"2023-07-14T12:00:06.814633+0000","last_peered":"2023-07-14T12:00:06.814633+0000","last_clean":"2023-07-14T12:00:06.814633+0000","last_became_active":"2023-07-14T11:59:47.782283+0000","last_became_peered":"2023-07-14T11:59:47.782283+0000","last_unstale":"2023-07-14T12:00:06.814633+0000","last_undegraded":"2023-07-14T12:00:06.814633+0000","last_fullsized":"2023-07-14T12:00:06.814633+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:50:29.299244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.809415+0000","last_change":"2023-07-14T11:59:42.756701+0000","last_active":"2023-07-14T12:00:06.809415+0000","last_peered":"2023-07-14T12:00:06.809415+0000","last_clean":"2023-07-14T12:00:06.809415+0000","last_became_active":"2023-07-14T11:59:42.756452+0000","last_became_peered":"2023-07-14T11:59:42.756452+0000","last_unstale":"2023-07-14T12:00:06.809415+0000","last_undegraded":"2023-07-14T12:00:06.809415+0000","last_fullsized":"2023-07-14T12:00:06.809415+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:56:37.979449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.804650+0000","last_change":"2023-07-14T11:59:43.457029+0000","last_active":"2023-07-14T12:00:06.804650+0000","last_peered":"2023-07-14T12:00:06.804650+0000","last_clean":"2023-07-14T12:00:06.804650+0000","last_became_active":"2023-07-14T11:59:43.456710+0000","last_became_peered":"2023-07-14T11:59:43.456710+0000","last_unstale":"2023-07-14T12:00:06.804650+0000","last_undegraded":"2023-07-14T12:00:06.804650+0000","last_fullsized":"2023-07-14T12:00:06.804650+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:17:39.969654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.811649+0000","last_change":"2023-07-14T11:59:47.785732+0000","last_active":"2023-07-14T12:00:06.811649+0000","last_peered":"2023-07-14T12:00:06.811649+0000","last_clean":"2023-07-14T12:00:06.811649+0000","last_became_active":"2023-07-14T11:59:47.785289+0000","last_became_peered":"2023-07-14T11:59:47.785289+0000","last_unstale":"2023-07-14T12:00:06.811649+0000","last_undegraded":"2023-07-14T12:00:06.811649+0000","last_fullsized":"2023-07-14T12:00:06.811649+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:58:47.873312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:13.463147+0000","last_change":"2023-07-14T11:59:42.760013+0000","last_active":"2023-07-14T12:00:13.463147+0000","last_peered":"2023-07-14T12:00:13.463147+0000","last_clean":"2023-07-14T12:00:13.463147+0000","last_became_active":"2023-07-14T11:59:42.759673+0000","last_became_peered":"2023-07-14T11:59:42.759673+0000","last_unstale":"2023-07-14T12:00:13.463147+0000","last_undegraded":"2023-07-14T12:00:13.463147+0000","last_fullsized":"2023-07-14T12:00:13.463147+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:31:23.656642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805922+0000","last_change":"2023-07-14T11:59:43.455418+0000","last_active":"2023-07-14T12:00:06.805922+0000","last_peered":"2023-07-14T12:00:06.805922+0000","last_clean":"2023-07-14T12:00:06.805922+0000","last_became_active":"2023-07-14T11:59:43.454963+0000","last_became_peered":"2023-07-14T11:59:43.454963+0000","last_unstale":"2023-07-14T12:00:06.805922+0000","last_undegraded":"2023-07-14T12:00:06.805922+0000","last_fullsized":"2023-07-14T12:00:06.805922+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:35:27.262942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805369+0000","last_change":"2023-07-14T11:59:47.784330+0000","last_active":"2023-07-14T12:00:06.805369+0000","last_peered":"2023-07-14T12:00:06.805369+0000","last_clean":"2023-07-14T12:00:06.805369+0000","last_became_active":"2023-07-14T11:59:47.783640+0000","last_became_peered":"2023-07-14T11:59:47.783640+0000","last_unstale":"2023-07-14T12:00:06.805369+0000","last_undegraded":"2023-07-14T12:00:06.805369+0000","last_fullsized":"2023-07-14T12:00:06.805369+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:23:46.613595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:13.465788+0000","last_change":"2023-07-14T11:59:42.758856+0000","last_active":"2023-07-14T12:00:13.465788+0000","last_peered":"2023-07-14T12:00:13.465788+0000","last_clean":"2023-07-14T12:00:13.465788+0000","last_became_active":"2023-07-14T11:59:42.758571+0000","last_became_peered":"2023-07-14T11:59:42.758571+0000","last_unstale":"2023-07-14T12:00:13.465788+0000","last_undegraded":"2023-07-14T12:00:13.465788+0000","last_fullsized":"2023-07-14T12:00:13.465788+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:22:26.607862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.814254+0000","last_change":"2023-07-14T11:59:43.466302+0000","last_active":"2023-07-14T12:00:06.814254+0000","last_peered":"2023-07-14T12:00:06.814254+0000","last_clean":"2023-07-14T12:00:06.814254+0000","last_became_active":"2023-07-14T11:59:43.465958+0000","last_became_peered":"2023-07-14T11:59:43.465958+0000","last_unstale":"2023-07-14T12:00:06.814254+0000","last_undegraded":"2023-07-14T12:00:06.814254+0000","last_fullsized":"2023-07-14T12:00:06.814254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:09:00.018902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.813066+0000","last_change":"2023-07-14T12:00:06.813066+0000","last_active":"2023-07-14T12:00:06.813066+0000","last_peered":"2023-07-14T12:00:06.813066+0000","last_clean":"2023-07-14T12:00:06.813066+0000","last_became_active":"2023-07-14T12:00:06.812460+0000","last_became_peered":"2023-07-14T12:00:06.812460+0000","last_unstale":"2023-07-14T12:00:06.813066+0000","last_undegraded":"2023-07-14T12:00:06.813066+0000","last_fullsized":"2023-07-14T12:00:06.813066+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:47:40.893162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808691+0000","last_change":"2023-07-14T11:59:42.765777+0000","last_active":"2023-07-14T12:00:06.808691+0000","last_peered":"2023-07-14T12:00:06.808691+0000","last_clean":"2023-07-14T12:00:06.808691+0000","last_became_active":"2023-07-14T11:59:42.765536+0000","last_became_peered":"2023-07-14T11:59:42.765536+0000","last_unstale":"2023-07-14T12:00:06.808691+0000","last_undegraded":"2023-07-14T12:00:06.808691+0000","last_fullsized":"2023-07-14T12:00:06.808691+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:27:05.053926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805426+0000","last_change":"2023-07-14T11:59:43.463665+0000","last_active":"2023-07-14T12:00:06.805426+0000","last_peered":"2023-07-14T12:00:06.805426+0000","last_clean":"2023-07-14T12:00:06.805426+0000","last_became_active":"2023-07-14T11:59:43.463368+0000","last_became_peered":"2023-07-14T11:59:43.463368+0000","last_unstale":"2023-07-14T12:00:06.805426+0000","last_undegraded":"2023-07-14T12:00:06.805426+0000","last_fullsized":"2023-07-14T12:00:06.805426+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:49:21.359682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.804608+0000","last_change":"2023-07-14T11:59:47.780706+0000","last_active":"2023-07-14T12:00:06.804608+0000","last_peered":"2023-07-14T12:00:06.804608+0000","last_clean":"2023-07-14T12:00:06.804608+0000","last_became_active":"2023-07-14T11:59:47.780207+0000","last_became_peered":"2023-07-14T11:59:47.780207+0000","last_unstale":"2023-07-14T12:00:06.804608+0000","last_undegraded":"2023-07-14T12:00:06.804608+0000","last_fullsized":"2023-07-14T12:00:06.804608+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:45:25.431762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'12","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:13.463278+0000","last_change":"2023-07-14T11:59:42.757238+0000","last_active":"2023-07-14T12:00:13.463278+0000","last_peered":"2023-07-14T12:00:13.463278+0000","last_clean":"2023-07-14T12:00:13.463278+0000","last_became_active":"2023-07-14T11:59:42.756887+0000","last_became_peered":"2023-07-14T11:59:42.756887+0000","last_unstale":"2023-07-14T12:00:13.463278+0000","last_undegraded":"2023-07-14T12:00:13.463278+0000","last_fullsized":"2023-07-14T12:00:13.463278+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:09:27.857990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3794,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808529+0000","last_change":"2023-07-14T11:59:43.460741+0000","last_active":"2023-07-14T12:00:06.808529+0000","last_peered":"2023-07-14T12:00:06.808529+0000","last_clean":"2023-07-14T12:00:06.808529+0000","last_became_active":"2023-07-14T11:59:43.460423+0000","last_became_peered":"2023-07-14T11:59:43.460423+0000","last_unstale":"2023-07-14T12:00:06.808529+0000","last_undegraded":"2023-07-14T12:00:06.808529+0000","last_fullsized":"2023-07-14T12:00:06.808529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:26:56.645695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.809254+0000","last_change":"2023-07-14T11:59:47.781321+0000","last_active":"2023-07-14T12:00:06.809254+0000","last_peered":"2023-07-14T12:00:06.809254+0000","last_clean":"2023-07-14T12:00:06.809254+0000","last_became_active":"2023-07-14T11:59:47.780941+0000","last_became_peered":"2023-07-14T11:59:47.780941+0000","last_unstale":"2023-07-14T12:00:06.809254+0000","last_undegraded":"2023-07-14T12:00:06.809254+0000","last_fullsized":"2023-07-14T12:00:06.809254+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:25:26.671430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.806651+0000","last_change":"2023-07-14T11:59:42.759403+0000","last_active":"2023-07-14T12:00:06.806651+0000","last_peered":"2023-07-14T12:00:06.806651+0000","last_clean":"2023-07-14T12:00:06.806651+0000","last_became_active":"2023-07-14T11:59:42.759036+0000","last_became_peered":"2023-07-14T11:59:42.759036+0000","last_unstale":"2023-07-14T12:00:06.806651+0000","last_undegraded":"2023-07-14T12:00:06.806651+0000","last_fullsized":"2023-07-14T12:00:06.806651+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:45:13.233768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808124+0000","last_change":"2023-07-14T11:59:43.457644+0000","last_active":"2023-07-14T12:00:06.808124+0000","last_peered":"2023-07-14T12:00:06.808124+0000","last_clean":"2023-07-14T12:00:06.808124+0000","last_became_active":"2023-07-14T11:59:43.457363+0000","last_became_peered":"2023-07-14T11:59:43.457363+0000","last_unstale":"2023-07-14T12:00:06.808124+0000","last_undegraded":"2023-07-14T12:00:06.808124+0000","last_fullsized":"2023-07-14T12:00:06.808124+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:12:17.222392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.812650+0000","last_change":"2023-07-14T11:59:47.789684+0000","last_active":"2023-07-14T12:00:06.812650+0000","last_peered":"2023-07-14T12:00:06.812650+0000","last_clean":"2023-07-14T12:00:06.812650+0000","last_became_active":"2023-07-14T11:59:47.789226+0000","last_became_peered":"2023-07-14T11:59:47.789226+0000","last_unstale":"2023-07-14T12:00:06.812650+0000","last_undegraded":"2023-07-14T12:00:06.812650+0000","last_fullsized":"2023-07-14T12:00:06.812650+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:56:58.961737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:13.466046+0000","last_change":"2023-07-14T11:59:42.751497+0000","last_active":"2023-07-14T12:00:13.466046+0000","last_peered":"2023-07-14T12:00:13.466046+0000","last_clean":"2023-07-14T12:00:13.466046+0000","last_became_active":"2023-07-14T11:59:42.751217+0000","last_became_peered":"2023-07-14T11:59:42.751217+0000","last_unstale":"2023-07-14T12:00:13.466046+0000","last_undegraded":"2023-07-14T12:00:13.466046+0000","last_fullsized":"2023-07-14T12:00:13.466046+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:53:00.140364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.328171+0000","last_change":"2023-07-14T11:59:43.462359+0000","last_active":"2023-07-14T12:00:07.328171+0000","last_peered":"2023-07-14T12:00:07.328171+0000","last_clean":"2023-07-14T12:00:07.328171+0000","last_became_active":"2023-07-14T11:59:43.461781+0000","last_became_peered":"2023-07-14T11:59:43.461781+0000","last_unstale":"2023-07-14T12:00:07.328171+0000","last_undegraded":"2023-07-14T12:00:07.328171+0000","last_fullsized":"2023-07-14T12:00:07.328171+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:10:22.527860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805028+0000","last_change":"2023-07-14T11:59:47.790173+0000","last_active":"2023-07-14T12:00:06.805028+0000","last_peered":"2023-07-14T12:00:06.805028+0000","last_clean":"2023-07-14T12:00:06.805028+0000","last_became_active":"2023-07-14T11:59:47.789418+0000","last_became_peered":"2023-07-14T11:59:47.789418+0000","last_unstale":"2023-07-14T12:00:06.805028+0000","last_undegraded":"2023-07-14T12:00:06.805028+0000","last_fullsized":"2023-07-14T12:00:06.805028+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:54:21.435699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:14.477363+0000","last_change":"2023-07-14T11:59:42.753068+0000","last_active":"2023-07-14T12:00:14.477363+0000","last_peered":"2023-07-14T12:00:14.477363+0000","last_clean":"2023-07-14T12:00:14.477363+0000","last_became_active":"2023-07-14T11:59:42.752794+0000","last_became_peered":"2023-07-14T11:59:42.752794+0000","last_unstale":"2023-07-14T12:00:14.477363+0000","last_undegraded":"2023-07-14T12:00:14.477363+0000","last_fullsized":"2023-07-14T12:00:14.477363+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:31:01.258740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.328012+0000","last_change":"2023-07-14T11:59:43.468655+0000","last_active":"2023-07-14T12:00:07.328012+0000","last_peered":"2023-07-14T12:00:07.328012+0000","last_clean":"2023-07-14T12:00:07.328012+0000","last_became_active":"2023-07-14T11:59:43.468361+0000","last_became_peered":"2023-07-14T11:59:43.468361+0000","last_unstale":"2023-07-14T12:00:07.328012+0000","last_undegraded":"2023-07-14T12:00:07.328012+0000","last_fullsized":"2023-07-14T12:00:07.328012+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:02:47.181093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808379+0000","last_change":"2023-07-14T11:59:47.783644+0000","last_active":"2023-07-14T12:00:06.808379+0000","last_peered":"2023-07-14T12:00:06.808379+0000","last_clean":"2023-07-14T12:00:06.808379+0000","last_became_active":"2023-07-14T11:59:47.783229+0000","last_became_peered":"2023-07-14T11:59:47.783229+0000","last_unstale":"2023-07-14T12:00:06.808379+0000","last_undegraded":"2023-07-14T12:00:06.808379+0000","last_fullsized":"2023-07-14T12:00:06.808379+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:15:35.788258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"83'8","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808245+0000","last_change":"2023-07-14T11:59:42.768482+0000","last_active":"2023-07-14T12:00:06.808245+0000","last_peered":"2023-07-14T12:00:06.808245+0000","last_clean":"2023-07-14T12:00:06.808245+0000","last_became_active":"2023-07-14T11:59:42.768201+0000","last_became_peered":"2023-07-14T11:59:42.768201+0000","last_unstale":"2023-07-14T12:00:06.808245+0000","last_undegraded":"2023-07-14T12:00:06.808245+0000","last_fullsized":"2023-07-14T12:00:06.808245+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:39:29.438053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808385+0000","last_change":"2023-07-14T11:59:43.469651+0000","last_active":"2023-07-14T12:00:06.808385+0000","last_peered":"2023-07-14T12:00:06.808385+0000","last_clean":"2023-07-14T12:00:06.808385+0000","last_became_active":"2023-07-14T11:59:43.465848+0000","last_became_peered":"2023-07-14T11:59:43.465848+0000","last_unstale":"2023-07-14T12:00:06.808385+0000","last_undegraded":"2023-07-14T12:00:06.808385+0000","last_fullsized":"2023-07-14T12:00:06.808385+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:17:18.313257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.817649+0000","last_change":"2023-07-14T12:00:06.817110+0000","last_active":"2023-07-14T12:00:06.817649+0000","last_peered":"2023-07-14T12:00:06.817649+0000","last_clean":"2023-07-14T12:00:06.817649+0000","last_became_active":"2023-07-14T12:00:06.816587+0000","last_became_peered":"2023-07-14T12:00:06.816587+0000","last_unstale":"2023-07-14T12:00:06.817649+0000","last_undegraded":"2023-07-14T12:00:06.817649+0000","last_fullsized":"2023-07-14T12:00:06.817649+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:04:52.873643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,0,3,1],"acting":[10,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.f","version":"83'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.807651+0000","last_change":"2023-07-14T11:59:42.758106+0000","last_active":"2023-07-14T12:00:06.807651+0000","last_peered":"2023-07-14T12:00:06.807651+0000","last_clean":"2023-07-14T12:00:06.807651+0000","last_became_active":"2023-07-14T11:59:42.757840+0000","last_became_peered":"2023-07-14T11:59:42.757840+0000","last_unstale":"2023-07-14T12:00:06.807651+0000","last_undegraded":"2023-07-14T12:00:06.807651+0000","last_fullsized":"2023-07-14T12:00:06.807651+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:12:46.164720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.812652+0000","last_change":"2023-07-14T11:59:43.467800+0000","last_active":"2023-07-14T12:00:06.812652+0000","last_peered":"2023-07-14T12:00:06.812652+0000","last_clean":"2023-07-14T12:00:06.812652+0000","last_became_active":"2023-07-14T11:59:43.467448+0000","last_became_peered":"2023-07-14T11:59:43.467448+0000","last_unstale":"2023-07-14T12:00:06.812652+0000","last_undegraded":"2023-07-14T12:00:06.812652+0000","last_fullsized":"2023-07-14T12:00:06.812652+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:34:25.781268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.807574+0000","last_change":"2023-07-14T11:59:47.915884+0000","last_active":"2023-07-14T12:00:06.807574+0000","last_peered":"2023-07-14T12:00:06.807574+0000","last_clean":"2023-07-14T12:00:06.807574+0000","last_became_active":"2023-07-14T11:59:47.915449+0000","last_became_peered":"2023-07-14T11:59:47.915449+0000","last_unstale":"2023-07-14T12:00:06.807574+0000","last_undegraded":"2023-07-14T12:00:06.807574+0000","last_fullsized":"2023-07-14T12:00:06.807574+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:41:55.813157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.804686+0000","last_change":"2023-07-14T11:59:42.771581+0000","last_active":"2023-07-14T12:00:06.804686+0000","last_peered":"2023-07-14T12:00:06.804686+0000","last_clean":"2023-07-14T12:00:06.804686+0000","last_became_active":"2023-07-14T11:59:42.771228+0000","last_became_peered":"2023-07-14T11:59:42.771228+0000","last_unstale":"2023-07-14T12:00:06.804686+0000","last_undegraded":"2023-07-14T12:00:06.804686+0000","last_fullsized":"2023-07-14T12:00:06.804686+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:32:35.711544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808751+0000","last_change":"2023-07-14T11:59:43.460987+0000","last_active":"2023-07-14T12:00:06.808751+0000","last_peered":"2023-07-14T12:00:06.808751+0000","last_clean":"2023-07-14T12:00:06.808751+0000","last_became_active":"2023-07-14T11:59:43.460604+0000","last_became_peered":"2023-07-14T11:59:43.460604+0000","last_unstale":"2023-07-14T12:00:06.808751+0000","last_undegraded":"2023-07-14T12:00:06.808751+0000","last_fullsized":"2023-07-14T12:00:06.808751+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:00:15.767598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808575+0000","last_change":"2023-07-14T11:59:47.788604+0000","last_active":"2023-07-14T12:00:06.808575+0000","last_peered":"2023-07-14T12:00:06.808575+0000","last_clean":"2023-07-14T12:00:06.808575+0000","last_became_active":"2023-07-14T11:59:47.787854+0000","last_became_peered":"2023-07-14T11:59:47.787854+0000","last_unstale":"2023-07-14T12:00:06.808575+0000","last_undegraded":"2023-07-14T12:00:06.808575+0000","last_fullsized":"2023-07-14T12:00:06.808575+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:19:18.510764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.328114+0000","last_change":"2023-07-14T11:59:42.768985+0000","last_active":"2023-07-14T12:00:07.328114+0000","last_peered":"2023-07-14T12:00:07.328114+0000","last_clean":"2023-07-14T12:00:07.328114+0000","last_became_active":"2023-07-14T11:59:42.768688+0000","last_became_peered":"2023-07-14T11:59:42.768688+0000","last_unstale":"2023-07-14T12:00:07.328114+0000","last_undegraded":"2023-07-14T12:00:07.328114+0000","last_fullsized":"2023-07-14T12:00:07.328114+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:20:55.661084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.807718+0000","last_change":"2023-07-14T11:59:43.465529+0000","last_active":"2023-07-14T12:00:06.807718+0000","last_peered":"2023-07-14T12:00:06.807718+0000","last_clean":"2023-07-14T12:00:06.807718+0000","last_became_active":"2023-07-14T11:59:43.465206+0000","last_became_peered":"2023-07-14T11:59:43.465206+0000","last_unstale":"2023-07-14T12:00:06.807718+0000","last_undegraded":"2023-07-14T12:00:06.807718+0000","last_fullsized":"2023-07-14T12:00:06.807718+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:52:33.760607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808309+0000","last_change":"2023-07-14T11:59:47.915842+0000","last_active":"2023-07-14T12:00:06.808309+0000","last_peered":"2023-07-14T12:00:06.808309+0000","last_clean":"2023-07-14T12:00:06.808309+0000","last_became_active":"2023-07-14T11:59:47.915304+0000","last_became_peered":"2023-07-14T11:59:47.915304+0000","last_unstale":"2023-07-14T12:00:06.808309+0000","last_undegraded":"2023-07-14T12:00:06.808309+0000","last_fullsized":"2023-07-14T12:00:06.808309+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:26:43.397241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.809764+0000","last_change":"2023-07-14T11:59:43.472296+0000","last_active":"2023-07-14T12:00:06.809764+0000","last_peered":"2023-07-14T12:00:06.809764+0000","last_clean":"2023-07-14T12:00:06.809764+0000","last_became_active":"2023-07-14T11:59:43.470569+0000","last_became_peered":"2023-07-14T11:59:43.470569+0000","last_unstale":"2023-07-14T12:00:06.809764+0000","last_undegraded":"2023-07-14T12:00:06.809764+0000","last_fullsized":"2023-07-14T12:00:06.809764+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:06:23.546910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.809241+0000","last_change":"2023-07-14T11:59:42.769225+0000","last_active":"2023-07-14T12:00:06.809241+0000","last_peered":"2023-07-14T12:00:06.809241+0000","last_clean":"2023-07-14T12:00:06.809241+0000","last_became_active":"2023-07-14T11:59:42.768877+0000","last_became_peered":"2023-07-14T11:59:42.768877+0000","last_unstale":"2023-07-14T12:00:06.809241+0000","last_undegraded":"2023-07-14T12:00:06.809241+0000","last_fullsized":"2023-07-14T12:00:06.809241+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:19:23.782648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.328375+0000","last_change":"2023-07-14T11:59:47.792665+0000","last_active":"2023-07-14T12:00:07.328375+0000","last_peered":"2023-07-14T12:00:07.328375+0000","last_clean":"2023-07-14T12:00:07.328375+0000","last_became_active":"2023-07-14T11:59:47.792308+0000","last_became_peered":"2023-07-14T11:59:47.792308+0000","last_unstale":"2023-07-14T12:00:07.328375+0000","last_undegraded":"2023-07-14T12:00:07.328375+0000","last_fullsized":"2023-07-14T12:00:07.328375+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:18:05.628049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.810583+0000","last_change":"2023-07-14T11:59:43.461103+0000","last_active":"2023-07-14T12:00:06.810583+0000","last_peered":"2023-07-14T12:00:06.810583+0000","last_clean":"2023-07-14T12:00:06.810583+0000","last_became_active":"2023-07-14T11:59:43.460756+0000","last_became_peered":"2023-07-14T11:59:43.460756+0000","last_unstale":"2023-07-14T12:00:06.810583+0000","last_undegraded":"2023-07-14T12:00:06.810583+0000","last_fullsized":"2023-07-14T12:00:06.810583+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:05:41.752001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:13.465872+0000","last_change":"2023-07-14T11:59:42.752680+0000","last_active":"2023-07-14T12:00:13.465872+0000","last_peered":"2023-07-14T12:00:13.465872+0000","last_clean":"2023-07-14T12:00:13.465872+0000","last_became_active":"2023-07-14T11:59:42.752305+0000","last_became_peered":"2023-07-14T11:59:42.752305+0000","last_unstale":"2023-07-14T12:00:13.465872+0000","last_undegraded":"2023-07-14T12:00:13.465872+0000","last_fullsized":"2023-07-14T12:00:13.465872+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:01:57.359059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":353,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.327820+0000","last_change":"2023-07-14T11:58:37.124871+0000","last_active":"2023-07-14T12:00:07.327820+0000","last_peered":"2023-07-14T12:00:07.327820+0000","last_clean":"2023-07-14T12:00:07.327820+0000","last_became_active":"2023-07-14T11:58:37.124398+0000","last_became_peered":"2023-07-14T11:58:37.124398+0000","last_unstale":"2023-07-14T12:00:07.327820+0000","last_undegraded":"2023-07-14T12:00:07.327820+0000","last_fullsized":"2023-07-14T12:00:07.327820+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:55:28.287160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:55:28.287160+0000","last_clean_scrub_stamp":"2023-07-14T11:55:28.287160+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:00:38.717405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.815687+0000","last_change":"2023-07-14T12:00:06.815687+0000","last_active":"2023-07-14T12:00:06.815687+0000","last_peered":"2023-07-14T12:00:06.815687+0000","last_clean":"2023-07-14T12:00:06.815687+0000","last_became_active":"2023-07-14T12:00:06.815281+0000","last_became_peered":"2023-07-14T12:00:06.815281+0000","last_unstale":"2023-07-14T12:00:06.815687+0000","last_undegraded":"2023-07-14T12:00:06.815687+0000","last_fullsized":"2023-07-14T12:00:06.815687+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:04:42.469562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.813952+0000","last_change":"2023-07-14T12:00:06.813952+0000","last_active":"2023-07-14T12:00:06.813952+0000","last_peered":"2023-07-14T12:00:06.813952+0000","last_clean":"2023-07-14T12:00:06.813952+0000","last_became_active":"2023-07-14T12:00:06.813581+0000","last_became_peered":"2023-07-14T12:00:06.813581+0000","last_unstale":"2023-07-14T12:00:06.813952+0000","last_undegraded":"2023-07-14T12:00:06.813952+0000","last_fullsized":"2023-07-14T12:00:06.813952+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:12:06.783097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.327974+0000","last_change":"2023-07-14T11:59:42.757400+0000","last_active":"2023-07-14T12:00:07.327974+0000","last_peered":"2023-07-14T12:00:07.327974+0000","last_clean":"2023-07-14T12:00:07.327974+0000","last_became_active":"2023-07-14T11:59:42.757047+0000","last_became_peered":"2023-07-14T11:59:42.757047+0000","last_unstale":"2023-07-14T12:00:07.327974+0000","last_undegraded":"2023-07-14T12:00:07.327974+0000","last_fullsized":"2023-07-14T12:00:07.327974+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:42:39.292073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.811396+0000","last_change":"2023-07-14T11:59:47.788486+0000","last_active":"2023-07-14T12:00:06.811396+0000","last_peered":"2023-07-14T12:00:06.811396+0000","last_clean":"2023-07-14T12:00:06.811396+0000","last_became_active":"2023-07-14T11:59:47.787995+0000","last_became_peered":"2023-07-14T11:59:47.787995+0000","last_unstale":"2023-07-14T12:00:06.811396+0000","last_undegraded":"2023-07-14T12:00:06.811396+0000","last_fullsized":"2023-07-14T12:00:06.811396+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:01:52.099573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808657+0000","last_change":"2023-07-14T11:59:42.765810+0000","last_active":"2023-07-14T12:00:06.808657+0000","last_peered":"2023-07-14T12:00:06.808657+0000","last_clean":"2023-07-14T12:00:06.808657+0000","last_became_active":"2023-07-14T11:59:42.765528+0000","last_became_peered":"2023-07-14T11:59:42.765528+0000","last_unstale":"2023-07-14T12:00:06.808657+0000","last_undegraded":"2023-07-14T12:00:06.808657+0000","last_fullsized":"2023-07-14T12:00:06.808657+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:18:02.321077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808724+0000","last_change":"2023-07-14T11:59:43.463763+0000","last_active":"2023-07-14T12:00:06.808724+0000","last_peered":"2023-07-14T12:00:06.808724+0000","last_clean":"2023-07-14T12:00:06.808724+0000","last_became_active":"2023-07-14T11:59:43.463411+0000","last_became_peered":"2023-07-14T11:59:43.463411+0000","last_unstale":"2023-07-14T12:00:06.808724+0000","last_undegraded":"2023-07-14T12:00:06.808724+0000","last_fullsized":"2023-07-14T12:00:06.808724+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:57:24.209856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.809720+0000","last_change":"2023-07-14T11:59:47.916405+0000","last_active":"2023-07-14T12:00:06.809720+0000","last_peered":"2023-07-14T12:00:06.809720+0000","last_clean":"2023-07-14T12:00:06.809720+0000","last_became_active":"2023-07-14T11:59:47.915892+0000","last_became_peered":"2023-07-14T11:59:47.915892+0000","last_unstale":"2023-07-14T12:00:06.809720+0000","last_undegraded":"2023-07-14T12:00:06.809720+0000","last_fullsized":"2023-07-14T12:00:06.809720+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:58:28.977188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"84'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805495+0000","last_change":"2023-07-14T11:59:42.754848+0000","last_active":"2023-07-14T12:00:06.805495+0000","last_peered":"2023-07-14T12:00:06.805495+0000","last_clean":"2023-07-14T12:00:06.805495+0000","last_became_active":"2023-07-14T11:59:42.754522+0000","last_became_peered":"2023-07-14T11:59:42.754522+0000","last_unstale":"2023-07-14T12:00:06.805495+0000","last_undegraded":"2023-07-14T12:00:06.805495+0000","last_fullsized":"2023-07-14T12:00:06.805495+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:49:48.659717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.815209+0000","last_change":"2023-07-14T11:59:43.458954+0000","last_active":"2023-07-14T12:00:06.815209+0000","last_peered":"2023-07-14T12:00:06.815209+0000","last_clean":"2023-07-14T12:00:06.815209+0000","last_became_active":"2023-07-14T11:59:43.458631+0000","last_became_peered":"2023-07-14T11:59:43.458631+0000","last_unstale":"2023-07-14T12:00:06.815209+0000","last_undegraded":"2023-07-14T12:00:06.815209+0000","last_fullsized":"2023-07-14T12:00:06.815209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:41:19.853895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805050+0000","last_change":"2023-07-14T11:59:47.776879+0000","last_active":"2023-07-14T12:00:06.805050+0000","last_peered":"2023-07-14T12:00:06.805050+0000","last_clean":"2023-07-14T12:00:06.805050+0000","last_became_active":"2023-07-14T11:59:47.776514+0000","last_became_peered":"2023-07-14T11:59:47.776514+0000","last_unstale":"2023-07-14T12:00:06.805050+0000","last_undegraded":"2023-07-14T12:00:06.805050+0000","last_fullsized":"2023-07-14T12:00:06.805050+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:14:32.460215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:13.464976+0000","last_change":"2023-07-14T11:59:42.754657+0000","last_active":"2023-07-14T12:00:13.464976+0000","last_peered":"2023-07-14T12:00:13.464976+0000","last_clean":"2023-07-14T12:00:13.464976+0000","last_became_active":"2023-07-14T11:59:42.754355+0000","last_became_peered":"2023-07-14T11:59:42.754355+0000","last_unstale":"2023-07-14T12:00:13.464976+0000","last_undegraded":"2023-07-14T12:00:13.464976+0000","last_fullsized":"2023-07-14T12:00:13.464976+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:16:59.044903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.329782+0000","last_change":"2023-07-14T12:00:07.329782+0000","last_active":"2023-07-14T12:00:07.329782+0000","last_peered":"2023-07-14T12:00:07.329782+0000","last_clean":"2023-07-14T12:00:07.329782+0000","last_became_active":"2023-07-14T12:00:07.329274+0000","last_became_peered":"2023-07-14T12:00:07.329274+0000","last_unstale":"2023-07-14T12:00:07.329782+0000","last_undegraded":"2023-07-14T12:00:07.329782+0000","last_fullsized":"2023-07-14T12:00:07.329782+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:49:01.837874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.807313+0000","last_change":"2023-07-14T11:59:47.916411+0000","last_active":"2023-07-14T12:00:06.807313+0000","last_peered":"2023-07-14T12:00:06.807313+0000","last_clean":"2023-07-14T12:00:06.807313+0000","last_became_active":"2023-07-14T11:59:47.915877+0000","last_became_peered":"2023-07-14T11:59:47.915877+0000","last_unstale":"2023-07-14T12:00:06.807313+0000","last_undegraded":"2023-07-14T12:00:06.807313+0000","last_fullsized":"2023-07-14T12:00:06.807313+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:08:05.471264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.804971+0000","last_change":"2023-07-14T11:59:42.769080+0000","last_active":"2023-07-14T12:00:06.804971+0000","last_peered":"2023-07-14T12:00:06.804971+0000","last_clean":"2023-07-14T12:00:06.804971+0000","last_became_active":"2023-07-14T11:59:42.768782+0000","last_became_peered":"2023-07-14T11:59:42.768782+0000","last_unstale":"2023-07-14T12:00:06.804971+0000","last_undegraded":"2023-07-14T12:00:06.804971+0000","last_fullsized":"2023-07-14T12:00:06.804971+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:31:48.128002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.806874+0000","last_change":"2023-07-14T11:59:43.460284+0000","last_active":"2023-07-14T12:00:06.806874+0000","last_peered":"2023-07-14T12:00:06.806874+0000","last_clean":"2023-07-14T12:00:06.806874+0000","last_became_active":"2023-07-14T11:59:43.459979+0000","last_became_peered":"2023-07-14T11:59:43.459979+0000","last_unstale":"2023-07-14T12:00:06.806874+0000","last_undegraded":"2023-07-14T12:00:06.806874+0000","last_fullsized":"2023-07-14T12:00:06.806874+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:25:15.513703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805201+0000","last_change":"2023-07-14T11:59:47.782359+0000","last_active":"2023-07-14T12:00:06.805201+0000","last_peered":"2023-07-14T12:00:06.805201+0000","last_clean":"2023-07-14T12:00:06.805201+0000","last_became_active":"2023-07-14T11:59:47.780484+0000","last_became_peered":"2023-07-14T11:59:47.780484+0000","last_unstale":"2023-07-14T12:00:06.805201+0000","last_undegraded":"2023-07-14T12:00:06.805201+0000","last_fullsized":"2023-07-14T12:00:06.805201+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:24:16.054488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.811311+0000","last_change":"2023-07-14T11:59:42.756823+0000","last_active":"2023-07-14T12:00:06.811311+0000","last_peered":"2023-07-14T12:00:06.811311+0000","last_clean":"2023-07-14T12:00:06.811311+0000","last_became_active":"2023-07-14T11:59:42.756412+0000","last_became_peered":"2023-07-14T11:59:42.756412+0000","last_unstale":"2023-07-14T12:00:06.811311+0000","last_undegraded":"2023-07-14T12:00:06.811311+0000","last_fullsized":"2023-07-14T12:00:06.811311+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:45:25.674809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.811156+0000","last_change":"2023-07-14T11:59:43.469151+0000","last_active":"2023-07-14T12:00:06.811156+0000","last_peered":"2023-07-14T12:00:06.811156+0000","last_clean":"2023-07-14T12:00:06.811156+0000","last_became_active":"2023-07-14T11:59:43.468835+0000","last_became_peered":"2023-07-14T11:59:43.468835+0000","last_unstale":"2023-07-14T12:00:06.811156+0000","last_undegraded":"2023-07-14T12:00:06.811156+0000","last_fullsized":"2023-07-14T12:00:06.811156+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:01:00.517226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.807655+0000","last_change":"2023-07-14T11:59:47.782663+0000","last_active":"2023-07-14T12:00:06.807655+0000","last_peered":"2023-07-14T12:00:06.807655+0000","last_clean":"2023-07-14T12:00:06.807655+0000","last_became_active":"2023-07-14T11:59:47.782175+0000","last_became_peered":"2023-07-14T11:59:47.782175+0000","last_unstale":"2023-07-14T12:00:06.807655+0000","last_undegraded":"2023-07-14T12:00:06.807655+0000","last_fullsized":"2023-07-14T12:00:06.807655+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:43:48.229057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805876+0000","last_change":"2023-07-14T11:59:42.759809+0000","last_active":"2023-07-14T12:00:06.805876+0000","last_peered":"2023-07-14T12:00:06.805876+0000","last_clean":"2023-07-14T12:00:06.805876+0000","last_became_active":"2023-07-14T11:59:42.759493+0000","last_became_peered":"2023-07-14T11:59:42.759493+0000","last_unstale":"2023-07-14T12:00:06.805876+0000","last_undegraded":"2023-07-14T12:00:06.805876+0000","last_fullsized":"2023-07-14T12:00:06.805876+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:21:22.391310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.810840+0000","last_change":"2023-07-14T11:59:43.465308+0000","last_active":"2023-07-14T12:00:06.810840+0000","last_peered":"2023-07-14T12:00:06.810840+0000","last_clean":"2023-07-14T12:00:06.810840+0000","last_became_active":"2023-07-14T11:59:43.464995+0000","last_became_peered":"2023-07-14T11:59:43.464995+0000","last_unstale":"2023-07-14T12:00:06.810840+0000","last_undegraded":"2023-07-14T12:00:06.810840+0000","last_fullsized":"2023-07-14T12:00:06.810840+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:22:41.954399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.807244+0000","last_change":"2023-07-14T11:59:47.916435+0000","last_active":"2023-07-14T12:00:06.807244+0000","last_peered":"2023-07-14T12:00:06.807244+0000","last_clean":"2023-07-14T12:00:06.807244+0000","last_became_active":"2023-07-14T11:59:47.915894+0000","last_became_peered":"2023-07-14T11:59:47.915894+0000","last_unstale":"2023-07-14T12:00:06.807244+0000","last_undegraded":"2023-07-14T12:00:06.807244+0000","last_fullsized":"2023-07-14T12:00:06.807244+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:15:10.755288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.807899+0000","last_change":"2023-07-14T11:59:42.760422+0000","last_active":"2023-07-14T12:00:06.807899+0000","last_peered":"2023-07-14T12:00:06.807899+0000","last_clean":"2023-07-14T12:00:06.807899+0000","last_became_active":"2023-07-14T11:59:42.758548+0000","last_became_peered":"2023-07-14T11:59:42.758548+0000","last_unstale":"2023-07-14T12:00:06.807899+0000","last_undegraded":"2023-07-14T12:00:06.807899+0000","last_fullsized":"2023-07-14T12:00:06.807899+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:00:42.799391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.328110+0000","last_change":"2023-07-14T11:59:43.455296+0000","last_active":"2023-07-14T12:00:07.328110+0000","last_peered":"2023-07-14T12:00:07.328110+0000","last_clean":"2023-07-14T12:00:07.328110+0000","last_became_active":"2023-07-14T11:59:43.455003+0000","last_became_peered":"2023-07-14T11:59:43.455003+0000","last_unstale":"2023-07-14T12:00:07.328110+0000","last_undegraded":"2023-07-14T12:00:07.328110+0000","last_fullsized":"2023-07-14T12:00:07.328110+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:43:29.555694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805023+0000","last_change":"2023-07-14T11:59:47.779262+0000","last_active":"2023-07-14T12:00:06.805023+0000","last_peered":"2023-07-14T12:00:06.805023+0000","last_clean":"2023-07-14T12:00:06.805023+0000","last_became_active":"2023-07-14T11:59:47.778863+0000","last_became_peered":"2023-07-14T11:59:47.778863+0000","last_unstale":"2023-07-14T12:00:06.805023+0000","last_undegraded":"2023-07-14T12:00:06.805023+0000","last_fullsized":"2023-07-14T12:00:06.805023+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:11:16.752296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.806219+0000","last_change":"2023-07-14T11:59:42.768252+0000","last_active":"2023-07-14T12:00:06.806219+0000","last_peered":"2023-07-14T12:00:06.806219+0000","last_clean":"2023-07-14T12:00:06.806219+0000","last_became_active":"2023-07-14T11:59:42.767956+0000","last_became_peered":"2023-07-14T11:59:42.767956+0000","last_unstale":"2023-07-14T12:00:06.806219+0000","last_undegraded":"2023-07-14T12:00:06.806219+0000","last_fullsized":"2023-07-14T12:00:06.806219+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:19:59.247002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.328734+0000","last_change":"2023-07-14T11:59:43.461899+0000","last_active":"2023-07-14T12:00:07.328734+0000","last_peered":"2023-07-14T12:00:07.328734+0000","last_clean":"2023-07-14T12:00:07.328734+0000","last_became_active":"2023-07-14T11:59:43.461535+0000","last_became_peered":"2023-07-14T11:59:43.461535+0000","last_unstale":"2023-07-14T12:00:07.328734+0000","last_undegraded":"2023-07-14T12:00:07.328734+0000","last_fullsized":"2023-07-14T12:00:07.328734+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:43:24.603371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.812720+0000","last_change":"2023-07-14T11:59:47.782964+0000","last_active":"2023-07-14T12:00:06.812720+0000","last_peered":"2023-07-14T12:00:06.812720+0000","last_clean":"2023-07-14T12:00:06.812720+0000","last_became_active":"2023-07-14T11:59:47.782563+0000","last_became_peered":"2023-07-14T11:59:47.782563+0000","last_unstale":"2023-07-14T12:00:06.812720+0000","last_undegraded":"2023-07-14T12:00:06.812720+0000","last_fullsized":"2023-07-14T12:00:06.812720+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:27:49.195128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808938+0000","last_change":"2023-07-14T11:59:42.751355+0000","last_active":"2023-07-14T12:00:06.808938+0000","last_peered":"2023-07-14T12:00:06.808938+0000","last_clean":"2023-07-14T12:00:06.808938+0000","last_became_active":"2023-07-14T11:59:42.751063+0000","last_became_peered":"2023-07-14T11:59:42.751063+0000","last_unstale":"2023-07-14T12:00:06.808938+0000","last_undegraded":"2023-07-14T12:00:06.808938+0000","last_fullsized":"2023-07-14T12:00:06.808938+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:28:15.966689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.328252+0000","last_change":"2023-07-14T11:59:43.465891+0000","last_active":"2023-07-14T12:00:07.328252+0000","last_peered":"2023-07-14T12:00:07.328252+0000","last_clean":"2023-07-14T12:00:07.328252+0000","last_became_active":"2023-07-14T11:59:43.465575+0000","last_became_peered":"2023-07-14T11:59:43.465575+0000","last_unstale":"2023-07-14T12:00:07.328252+0000","last_undegraded":"2023-07-14T12:00:07.328252+0000","last_fullsized":"2023-07-14T12:00:07.328252+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:43:52.704477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.809069+0000","last_change":"2023-07-14T11:59:47.777547+0000","last_active":"2023-07-14T12:00:06.809069+0000","last_peered":"2023-07-14T12:00:06.809069+0000","last_clean":"2023-07-14T12:00:06.809069+0000","last_became_active":"2023-07-14T11:59:47.777150+0000","last_became_peered":"2023-07-14T11:59:47.777150+0000","last_unstale":"2023-07-14T12:00:06.809069+0000","last_undegraded":"2023-07-14T12:00:06.809069+0000","last_fullsized":"2023-07-14T12:00:06.809069+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:21:35.638026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.809105+0000","last_change":"2023-07-14T11:59:42.761236+0000","last_active":"2023-07-14T12:00:06.809105+0000","last_peered":"2023-07-14T12:00:06.809105+0000","last_clean":"2023-07-14T12:00:06.809105+0000","last_became_active":"2023-07-14T11:59:42.760893+0000","last_became_peered":"2023-07-14T11:59:42.760893+0000","last_unstale":"2023-07-14T12:00:06.809105+0000","last_undegraded":"2023-07-14T12:00:06.809105+0000","last_fullsized":"2023-07-14T12:00:06.809105+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:11:17.477118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.328670+0000","last_change":"2023-07-14T11:59:43.462518+0000","last_active":"2023-07-14T12:00:07.328670+0000","last_peered":"2023-07-14T12:00:07.328670+0000","last_clean":"2023-07-14T12:00:07.328670+0000","last_became_active":"2023-07-14T11:59:43.462178+0000","last_became_peered":"2023-07-14T11:59:43.462178+0000","last_unstale":"2023-07-14T12:00:07.328670+0000","last_undegraded":"2023-07-14T12:00:07.328670+0000","last_fullsized":"2023-07-14T12:00:07.328670+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:09:38.398771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.811302+0000","last_change":"2023-07-14T11:59:43.474142+0000","last_active":"2023-07-14T12:00:06.811302+0000","last_peered":"2023-07-14T12:00:06.811302+0000","last_clean":"2023-07-14T12:00:06.811302+0000","last_became_active":"2023-07-14T11:59:43.473863+0000","last_became_peered":"2023-07-14T11:59:43.473863+0000","last_unstale":"2023-07-14T12:00:06.811302+0000","last_undegraded":"2023-07-14T12:00:06.811302+0000","last_fullsized":"2023-07-14T12:00:06.811302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:34:40.849796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805367+0000","last_change":"2023-07-14T11:59:43.462193+0000","last_active":"2023-07-14T12:00:06.805367+0000","last_peered":"2023-07-14T12:00:06.805367+0000","last_clean":"2023-07-14T12:00:06.805367+0000","last_became_active":"2023-07-14T11:59:43.461390+0000","last_became_peered":"2023-07-14T11:59:43.461390+0000","last_unstale":"2023-07-14T12:00:06.805367+0000","last_undegraded":"2023-07-14T12:00:06.805367+0000","last_fullsized":"2023-07-14T12:00:06.805367+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:43:44.553075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805098+0000","last_change":"2023-07-14T11:59:43.467561+0000","last_active":"2023-07-14T12:00:06.805098+0000","last_peered":"2023-07-14T12:00:06.805098+0000","last_clean":"2023-07-14T12:00:06.805098+0000","last_became_active":"2023-07-14T11:59:43.467256+0000","last_became_peered":"2023-07-14T11:59:43.467256+0000","last_unstale":"2023-07-14T12:00:06.805098+0000","last_undegraded":"2023-07-14T12:00:06.805098+0000","last_fullsized":"2023-07-14T12:00:06.805098+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:04:06.326191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.813046+0000","last_change":"2023-07-14T11:59:43.472405+0000","last_active":"2023-07-14T12:00:06.813046+0000","last_peered":"2023-07-14T12:00:06.813046+0000","last_clean":"2023-07-14T12:00:06.813046+0000","last_became_active":"2023-07-14T11:59:43.471863+0000","last_became_peered":"2023-07-14T11:59:43.471863+0000","last_unstale":"2023-07-14T12:00:06.813046+0000","last_undegraded":"2023-07-14T12:00:06.813046+0000","last_fullsized":"2023-07-14T12:00:06.813046+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:22:54.987390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808865+0000","last_change":"2023-07-14T11:59:43.469966+0000","last_active":"2023-07-14T12:00:06.808865+0000","last_peered":"2023-07-14T12:00:06.808865+0000","last_clean":"2023-07-14T12:00:06.808865+0000","last_became_active":"2023-07-14T11:59:43.469540+0000","last_became_peered":"2023-07-14T11:59:43.469540+0000","last_unstale":"2023-07-14T12:00:06.808865+0000","last_undegraded":"2023-07-14T12:00:06.808865+0000","last_fullsized":"2023-07-14T12:00:06.808865+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:19:04.529881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.812535+0000","last_change":"2023-07-14T11:59:43.465032+0000","last_active":"2023-07-14T12:00:06.812535+0000","last_peered":"2023-07-14T12:00:06.812535+0000","last_clean":"2023-07-14T12:00:06.812535+0000","last_became_active":"2023-07-14T11:59:43.464689+0000","last_became_peered":"2023-07-14T11:59:43.464689+0000","last_unstale":"2023-07-14T12:00:06.812535+0000","last_undegraded":"2023-07-14T12:00:06.812535+0000","last_fullsized":"2023-07-14T12:00:06.812535+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:15:12.845214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.806463+0000","last_change":"2023-07-14T11:59:43.463042+0000","last_active":"2023-07-14T12:00:06.806463+0000","last_peered":"2023-07-14T12:00:06.806463+0000","last_clean":"2023-07-14T12:00:06.806463+0000","last_became_active":"2023-07-14T11:59:43.462665+0000","last_became_peered":"2023-07-14T11:59:43.462665+0000","last_unstale":"2023-07-14T12:00:06.806463+0000","last_undegraded":"2023-07-14T12:00:06.806463+0000","last_fullsized":"2023-07-14T12:00:06.806463+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:50:19.141988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.816236+0000","last_change":"2023-07-14T12:00:06.816236+0000","last_active":"2023-07-14T12:00:06.816236+0000","last_peered":"2023-07-14T12:00:06.816236+0000","last_clean":"2023-07-14T12:00:06.816236+0000","last_became_active":"2023-07-14T12:00:06.815733+0000","last_became_peered":"2023-07-14T12:00:06.815733+0000","last_unstale":"2023-07-14T12:00:06.816236+0000","last_undegraded":"2023-07-14T12:00:06.816236+0000","last_fullsized":"2023-07-14T12:00:06.816236+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:14:57.114402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.809719+0000","last_change":"2023-07-14T11:59:43.470256+0000","last_active":"2023-07-14T12:00:06.809719+0000","last_peered":"2023-07-14T12:00:06.809719+0000","last_clean":"2023-07-14T12:00:06.809719+0000","last_became_active":"2023-07-14T11:59:43.469861+0000","last_became_peered":"2023-07-14T11:59:43.469861+0000","last_unstale":"2023-07-14T12:00:06.809719+0000","last_undegraded":"2023-07-14T12:00:06.809719+0000","last_fullsized":"2023-07-14T12:00:06.809719+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:54:04.221673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.811456+0000","last_change":"2023-07-14T11:59:43.458009+0000","last_active":"2023-07-14T12:00:06.811456+0000","last_peered":"2023-07-14T12:00:06.811456+0000","last_clean":"2023-07-14T12:00:06.811456+0000","last_became_active":"2023-07-14T11:59:43.457657+0000","last_became_peered":"2023-07-14T11:59:43.457657+0000","last_unstale":"2023-07-14T12:00:06.811456+0000","last_undegraded":"2023-07-14T12:00:06.811456+0000","last_fullsized":"2023-07-14T12:00:06.811456+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:14:41.854547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805358+0000","last_change":"2023-07-14T11:59:43.457568+0000","last_active":"2023-07-14T12:00:06.805358+0000","last_peered":"2023-07-14T12:00:06.805358+0000","last_clean":"2023-07-14T12:00:06.805358+0000","last_became_active":"2023-07-14T11:59:43.457203+0000","last_became_peered":"2023-07-14T11:59:43.457203+0000","last_unstale":"2023-07-14T12:00:06.805358+0000","last_undegraded":"2023-07-14T12:00:06.805358+0000","last_fullsized":"2023-07-14T12:00:06.805358+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:14:37.939509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808045+0000","last_change":"2023-07-14T11:59:43.470840+0000","last_active":"2023-07-14T12:00:06.808045+0000","last_peered":"2023-07-14T12:00:06.808045+0000","last_clean":"2023-07-14T12:00:06.808045+0000","last_became_active":"2023-07-14T11:59:43.470531+0000","last_became_peered":"2023-07-14T11:59:43.470531+0000","last_unstale":"2023-07-14T12:00:06.808045+0000","last_undegraded":"2023-07-14T12:00:06.808045+0000","last_fullsized":"2023-07-14T12:00:06.808045+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:49:50.862808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.817057+0000","last_change":"2023-07-14T12:00:06.817057+0000","last_active":"2023-07-14T12:00:06.817057+0000","last_peered":"2023-07-14T12:00:06.817057+0000","last_clean":"2023-07-14T12:00:06.817057+0000","last_became_active":"2023-07-14T12:00:06.816680+0000","last_became_peered":"2023-07-14T12:00:06.816680+0000","last_unstale":"2023-07-14T12:00:06.817057+0000","last_undegraded":"2023-07-14T12:00:06.817057+0000","last_fullsized":"2023-07-14T12:00:06.817057+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:22:01.363770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284390,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6412,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":16932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416401,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":970752,"data_stored":486215,"data_compressed":17563,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:59:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.50700000000000001}]},{"osd":1,"last update":"Fri Jul 14 11:59:56 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.71399999999999997}]},{"osd":2,"last update":"Fri Jul 14 11:59:56 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.377}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":4,"last update":"Fri Jul 14 11:59:56 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.61899999999999999}]},{"osd":5,"last update":"Fri Jul 14 11:59:56 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.54200000000000004}]},{"osd":6,"last update":"Fri Jul 14 11:59:56 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":2.1869999999999998,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":0.65200000000000002}]},{"osd":7,"last update":"Fri Jul 14 11:59:56 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.79000000000000004}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":9,"last update":"Fri Jul 14 11:59:56 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":2.0339999999999998,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.83099999999999996}]},{"osd":10,"last update":"Fri Jul 14 11:59:56 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":2.1019999999999999,"5min":2.1019999999999999,"15min":2.1019999999999999},"last":0.68200000000000005}]}]},{"osd":10,"up_from":72,"seq":309237645333,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":1016,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":1040384,"data_stored":1080240,"data_compressed":53729,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":1,"last update":"Fri Jul 14 11:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.435},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.48499999999999999}]},{"osd":2,"last update":"Fri Jul 14 11:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.628},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.67700000000000005}]},{"osd":3,"last update":"Fri Jul 14 11:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.58399999999999996}]},{"osd":4,"last update":"Fri Jul 14 11:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.59899999999999998}]},{"osd":5,"last update":"Fri Jul 14 11:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.61299999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.86199999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.60899999999999999}]},{"osd":8,"last update":"Fri Jul 14 11:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.75900000000000001}]},{"osd":9,"last update":"Fri Jul 14 11:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.69999999999999996}]},{"osd":11,"last update":"Fri Jul 14 11:59:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.79900000000000004}]}]},{"osd":9,"up_from":67,"seq":287762808858,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":974848,"data_stored":486797,"data_compressed":17563,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.85999999999999999}]},{"osd":1,"last update":"Fri Jul 14 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.96599999999999997}]},{"osd":2,"last update":"Fri Jul 14 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.91100000000000003}]},{"osd":3,"last update":"Fri Jul 14 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.88700000000000001}]},{"osd":4,"last update":"Fri Jul 14 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":2.0099999999999998,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":0.93500000000000005}]},{"osd":5,"last update":"Fri Jul 14 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.754},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.77800000000000002}]},{"osd":6,"last update":"Fri Jul 14 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":1.006}]},{"osd":7,"last update":"Fri Jul 14 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.95399999999999996}]},{"osd":8,"last update":"Fri Jul 14 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":1.105},{"interface":"front","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":1.079}]},{"osd":10,"last update":"Fri Jul 14 11:59:35 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":1.2130000000000001},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":1.1950000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1759999999999999}]}]},{"osd":8,"up_from":59,"seq":253403070495,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":978944,"data_stored":486819,"data_compressed":17563,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:59:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.60599999999999998,"15min":0.60999999999999999},"min":{"1min":0.42299999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.877,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.57899999999999996,"15min":0.58499999999999996},"min":{"1min":0.376,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81299999999999994,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.60599999999999998}]},{"osd":1,"last update":"Fri Jul 14 11:59:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58699999999999997,"15min":0.58099999999999996},"min":{"1min":0.41599999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.73099999999999998,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.63,"5min":0.627,"15min":0.627},"min":{"1min":0.437,"5min":0.376,"15min":0.376},"max":{"1min":0.80900000000000005,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.75}]},{"osd":2,"last update":"Fri Jul 14 11:59:49 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.34499999999999997,"5min":0.309,"15min":0.309},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.51100000000000001,"15min":0.51800000000000002},"min":{"1min":0.317,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.60699999999999998,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.54600000000000004}]},{"osd":3,"last update":"Fri Jul 14 11:59:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.63500000000000001,"15min":0.64300000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.878,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.65000000000000002,"15min":0.65400000000000003},"min":{"1min":0.44500000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.627}]},{"osd":4,"last update":"Fri Jul 14 11:59:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.61599999999999999,"15min":0.62},"min":{"1min":0.48399999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.77700000000000002,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.64900000000000002,"15min":0.64500000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84599999999999997,"5min":0.87,"15min":0.87},"last":0.76300000000000001}]},{"osd":5,"last update":"Fri Jul 14 11:59:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.61499999999999999,"15min":0.62},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.79600000000000004,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.37,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72599999999999998,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.51800000000000002}]},{"osd":6,"last update":"Fri Jul 14 11:59:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.74199999999999999,"15min":0.749},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.874,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.748,"5min":0.78700000000000003,"15min":0.79300000000000004},"min":{"1min":0.52300000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.92600000000000005,"5min":1.087,"15min":1.087},"last":0.81699999999999995}]},{"osd":7,"last update":"Fri Jul 14 11:59:49 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.751,"15min":0.753},"min":{"1min":0.502,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.99199999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.73799999999999999,"15min":0.746},"min":{"1min":0.51400000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.89300000000000002,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.79700000000000004}]},{"osd":9,"last update":"Fri Jul 14 11:59:17 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.878}]},{"osd":10,"last update":"Fri Jul 14 11:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.86499999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]}]},{"osd":7,"up_from":52,"seq":223338299428,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":1004,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":1028096,"data_stored":1077408,"data_compressed":53729,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.443,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.88300000000000001,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59699999999999998,"15min":0.59599999999999997},"min":{"1min":0.50700000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.495}]},{"osd":1,"last update":"Fri Jul 14 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.42099999999999999,"15min":0.41499999999999998},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.70199999999999996,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.439,"5min":0.44700000000000001,"15min":0.44800000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.41599999999999998}]},{"osd":2,"last update":"Fri Jul 14 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59499999999999997,"15min":0.59699999999999998},"min":{"1min":0.433,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77500000000000002,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.625,"15min":0.61699999999999999},"min":{"1min":0.48999999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.70299999999999996}]},{"osd":3,"last update":"Fri Jul 14 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.63200000000000001,"15min":0.629},"min":{"1min":0.47099999999999997,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.66000000000000003,"15min":0.65800000000000003},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.56299999999999994}]},{"osd":4,"last update":"Fri Jul 14 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.51000000000000001,"15min":0.501},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.52700000000000002,"15min":0.53100000000000003},"min":{"1min":0.35999999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.72799999999999998,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.5}]},{"osd":5,"last update":"Fri Jul 14 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.71799999999999997,"15min":0.72199999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.98899999999999999,"5min":1.341,"15min":1.341},"last":0.79900000000000004}]},{"osd":6,"last update":"Fri Jul 14 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.72199999999999998,"15min":0.71999999999999997},"min":{"1min":0.53100000000000003,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.69499999999999995,"15min":0.69699999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.85799999999999998,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.65700000000000003}]},{"osd":8,"last update":"Fri Jul 14 11:59:50 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.71199999999999997,"15min":0.69999999999999996},"min":{"1min":0.59999999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.70699999999999996,"15min":0.70099999999999996},"min":{"1min":0.57699999999999996,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.88100000000000001,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.76300000000000001}]},{"osd":9,"last update":"Fri Jul 14 11:59:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.72699999999999998}]},{"osd":10,"last update":"Fri Jul 14 11:59:35 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.77500000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]}]},{"osd":6,"up_from":47,"seq":201863462951,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":970752,"data_stored":486215,"data_compressed":17563,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 12:00:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.48699999999999999,"15min":0.47299999999999998},"min":{"1min":0.39100000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.60599999999999998,"15min":0.59799999999999998},"min":{"1min":0.46600000000000003,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.82299999999999995}]},{"osd":1,"last update":"Fri Jul 14 12:00:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.65600000000000003,"15min":0.64600000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":1.056},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.72699999999999998,"15min":0.72199999999999998},"min":{"1min":0.51200000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":1.109}]},{"osd":2,"last update":"Fri Jul 14 12:00:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.65500000000000003,"15min":0.64500000000000002},"min":{"1min":0.48999999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.93799999999999994,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.70099999999999996,"15min":0.68999999999999995},"min":{"1min":0.41399999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0049999999999999,"5min":1.155,"15min":1.155},"last":0.92500000000000004}]},{"osd":3,"last update":"Fri Jul 14 12:00:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.61399999999999999,"15min":0.59999999999999998},"min":{"1min":0.46200000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.93600000000000005,"5min":1.107,"15min":1.107},"last":0.75},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.70899999999999996,"15min":0.69699999999999995},"min":{"1min":0.57499999999999996,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91500000000000004,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":1.0329999999999999}]},{"osd":4,"last update":"Fri Jul 14 12:00:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.68500000000000005,"15min":0.67300000000000004},"min":{"1min":0.441,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.754,"15min":0.73999999999999999},"min":{"1min":0.57199999999999995,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":1.1519999999999999}]},{"osd":5,"last update":"Fri Jul 14 12:00:06 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.79900000000000004,"15min":0.79300000000000004},"min":{"1min":0.67600000000000005,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1000000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.745,"15min":0.73399999999999999},"min":{"1min":0.55000000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":1.0049999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.77100000000000002,"15min":0.752},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":1.169},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.73599999999999999,"15min":0.71499999999999997},"min":{"1min":0.67000000000000004,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0629999999999999,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":1.179}]},{"osd":8,"last update":"Fri Jul 14 11:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.81000000000000005,"15min":0.80300000000000005},"min":{"1min":0.63400000000000001,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":1.2030000000000001},{"interface":"front","average":{"1min":0.876,"5min":0.82099999999999995,"15min":0.81200000000000006},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.052,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":1.127}]},{"osd":9,"last update":"Fri Jul 14 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":1.23},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":1.2569999999999999}]},{"osd":10,"last update":"Fri Jul 14 11:59:35 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":1.218},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":1.278}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3300000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":983040,"data_stored":490009,"data_compressed":17563,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:59:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.55100000000000005,"15min":0.52900000000000003},"min":{"1min":0.41499999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.65400000000000003,"15min":0.73999999999999999},"min":{"1min":0.39300000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.90200000000000002,"5min":5.0940000000000003,"15min":5.0940000000000003},"last":0.56299999999999994}]},{"osd":2,"last update":"Fri Jul 14 11:59:37 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.53000000000000003,"15min":0.498},"min":{"1min":0.41899999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.64300000000000002,"15min":0.70299999999999996},"min":{"1min":0.47299999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.82499999999999996,"5min":5.0570000000000004,"15min":5.0570000000000004},"last":0.72999999999999998}]},{"osd":3,"last update":"Fri Jul 14 12:00:00 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60699999999999998,"15min":0.59699999999999998},"min":{"1min":0.39900000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.95999999999999996,"5min":1.054,"15min":1.054},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.42399999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.92600000000000005,"5min":1.006,"15min":1.006},"last":0.67500000000000004}]},{"osd":4,"last update":"Fri Jul 14 11:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.437,"15min":0.42099999999999999},"min":{"1min":0.36699999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1,"5min":1,"15min":1},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.46000000000000002,"15min":0.45100000000000001},"min":{"1min":0.34999999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.61199999999999999}]},{"osd":5,"last update":"Fri Jul 14 11:59:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.60899999999999999,"15min":0.58499999999999996},"min":{"1min":0.50900000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.83399999999999996,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.58399999999999996,"15min":0.55700000000000005},"min":{"1min":0.53600000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.81100000000000005,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.75700000000000001}]},{"osd":6,"last update":"Fri Jul 14 12:00:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.65700000000000003,"15min":0.63900000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.69399999999999995,"15min":0.67500000000000004},"min":{"1min":0.53800000000000003,"5min":0.44,"15min":0.44},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.77200000000000002}]},{"osd":7,"last update":"Fri Jul 14 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.54300000000000004,"15min":0.53200000000000003},"min":{"1min":0.39700000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.61799999999999999,"15min":0.59399999999999997},"min":{"1min":0.59899999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.85599999999999998}]},{"osd":8,"last update":"Fri Jul 14 11:59:49 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.71199999999999997,"15min":0.70099999999999996},"min":{"1min":0.624,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68999999999999995,"15min":0.68600000000000005},"min":{"1min":0.52700000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.84399999999999997}]},{"osd":9,"last update":"Fri Jul 14 11:59:16 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.746}]},{"osd":10,"last update":"Fri Jul 14 11:59:37 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":0.82699999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738435,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":974848,"data_stored":486305,"data_compressed":17563,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 11:59:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56399999999999995,"15min":0.54200000000000004},"min":{"1min":0.33100000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.96399999999999997,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.55000000000000004,"15min":0.52800000000000002},"min":{"1min":0.436,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.83299999999999996}]},{"osd":2,"last update":"Fri Jul 14 11:59:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.61699999999999999,"15min":0.58699999999999997},"min":{"1min":0.43099999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.0569999999999999,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.59299999999999997,"15min":0.56100000000000005},"min":{"1min":0.42199999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.84699999999999998,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.60299999999999998}]},{"osd":3,"last update":"Fri Jul 14 11:58:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.54500000000000004,"15min":0.52300000000000002},"min":{"1min":0.313,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.55700000000000005,"15min":0.51100000000000001},"min":{"1min":0.39600000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.375}]},{"osd":4,"last update":"Fri Jul 14 11:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.60799999999999998,"15min":0.58199999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.62,"15min":0.58499999999999996},"min":{"1min":0.40999999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.93200000000000005}]},{"osd":5,"last update":"Fri Jul 14 11:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.65900000000000003,"15min":0.64600000000000002},"min":{"1min":0.40500000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.96899999999999997,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.69599999999999995,"15min":0.67600000000000005},"min":{"1min":0.5,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.99099999999999999,"5min":1.131,"15min":1.131},"last":0.77800000000000002}]},{"osd":6,"last update":"Fri Jul 14 12:00:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.67600000000000005,"15min":0.66000000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.92600000000000005,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.68700000000000006,"15min":0.68100000000000005},"min":{"1min":0.40999999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86899999999999999,"5min":1.081,"15min":1.081},"last":0.70799999999999996}]},{"osd":7,"last update":"Fri Jul 14 11:59:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.72399999999999998,"15min":0.70999999999999996},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.88},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.75900000000000001,"15min":0.746},"min":{"1min":0.47199999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.90100000000000002}]},{"osd":8,"last update":"Fri Jul 14 11:59:48 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.78800000000000003,"15min":0.79600000000000004},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.93200000000000005,"5min":1.05,"15min":1.05},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81499999999999995,"15min":0.81599999999999995},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.80100000000000005}]},{"osd":9,"last update":"Fri Jul 14 11:59:13 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.97299999999999998}]},{"osd":10,"last update":"Fri Jul 14 11:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":1.008},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":1.0389999999999999}]},{"osd":11,"last update":"Fri Jul 14 11:59:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":1.115},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":1.1040000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":987136,"data_stored":490043,"data_compressed":17563,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.55300000000000005,"15min":0.501},"min":{"1min":0.39500000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.745,"5min":6.54,"15min":6.54},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.52100000000000002,"15min":0.497},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78100000000000003,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.56699999999999995}]},{"osd":1,"last update":"Fri Jul 14 11:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.52300000000000002,"15min":0.49199999999999999},"min":{"1min":0.39600000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.746},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.53500000000000003,"15min":0.5},"min":{"1min":0.373,"5min":0.31,"15min":0.31},"max":{"1min":0.91200000000000003,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.67800000000000005}]},{"osd":3,"last update":"Fri Jul 14 12:00:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.58499999999999996,"15min":0.54600000000000004},"min":{"1min":0.39200000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.83299999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.502},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.59599999999999997,"15min":0.55900000000000005},"min":{"1min":0.41599999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.88400000000000001,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.65400000000000003}]},{"osd":4,"last update":"Fri Jul 14 11:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.55000000000000004,"15min":0.53200000000000003},"min":{"1min":0.43099999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.55400000000000005,"15min":0.54300000000000004},"min":{"1min":0.39600000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.82999999999999996,"5min":0.873,"15min":0.873},"last":0.70899999999999996}]},{"osd":5,"last update":"Fri Jul 14 11:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.44400000000000001,"15min":0.435},"min":{"1min":0.313,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.46500000000000002,"15min":0.45000000000000001},"min":{"1min":0.28199999999999997,"5min":0.252,"15min":0.252},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.47099999999999997}]},{"osd":6,"last update":"Fri Jul 14 12:00:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.65200000000000002,"15min":0.63700000000000001},"min":{"1min":0.50700000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.64800000000000002,"15min":0.63800000000000001},"min":{"1min":0.46600000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.72899999999999998}]},{"osd":7,"last update":"Fri Jul 14 11:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.60999999999999999,"15min":0.60599999999999998},"min":{"1min":0.42799999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.79000000000000004,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.66800000000000004,"15min":0.67200000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.89900000000000002,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.79500000000000004}]},{"osd":8,"last update":"Fri Jul 14 11:59:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.59199999999999997,"15min":0.59799999999999998},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.93500000000000005,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.621,"15min":0.621},"min":{"1min":0.371,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.63300000000000001}]},{"osd":9,"last update":"Fri Jul 14 11:59:17 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.86599999999999999}]},{"osd":10,"last update":"Fri Jul 14 11:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.78100000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]}]},{"osd":3,"up_from":26,"seq":111669149750,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":978944,"data_stored":486319,"data_compressed":17563,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:59:56 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.42699999999999999,"15min":0.377},"min":{"1min":0.32900000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.45200000000000001,"15min":0.40100000000000002},"min":{"1min":0.32800000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.64200000000000002,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.40699999999999997}]},{"osd":1,"last update":"Fri Jul 14 11:59:56 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.64200000000000002,"15min":0.57299999999999995},"min":{"1min":0.44600000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0249999999999999,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.58499999999999996,"15min":0.54700000000000004},"min":{"1min":0.40000000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.94699999999999995,"5min":1.111,"15min":1.111},"last":0.78600000000000003}]},{"osd":2,"last update":"Fri Jul 14 11:59:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.65400000000000003,"15min":0.59999999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.89300000000000002,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.753},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.65500000000000003,"15min":0.60699999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.94399999999999995,"5min":1.206,"15min":1.206},"last":0.76000000000000001}]},{"osd":4,"last update":"Fri Jul 14 11:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.63600000000000001,"15min":0.60399999999999998},"min":{"1min":0.437,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.65300000000000002,"15min":0.61199999999999999},"min":{"1min":0.48299999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0860000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.89900000000000002}]},{"osd":5,"last update":"Fri Jul 14 11:59:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64100000000000001,"15min":0.623},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.89400000000000002,"5min":1.05,"15min":1.05},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.71799999999999997,"15min":0.68400000000000005},"min":{"1min":0.52300000000000002,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0189999999999999,"5min":1.077,"15min":1.077},"last":0.71099999999999997}]},{"osd":6,"last update":"Fri Jul 14 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.749,"15min":0.71799999999999997},"min":{"1min":0.53300000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.97999999999999998,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.69499999999999995,"15min":0.69399999999999995},"min":{"1min":0.44700000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.95699999999999996,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.56399999999999995}]},{"osd":7,"last update":"Fri Jul 14 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.86199999999999999,"15min":0.86599999999999999},"min":{"1min":0.62,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.113,"5min":1.254,"15min":1.254},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.89300000000000002,"15min":0.88800000000000001},"min":{"1min":0.64000000000000001,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.173,"5min":1.383,"15min":1.383},"last":0.94099999999999995}]},{"osd":8,"last update":"Fri Jul 14 11:58:46 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.82799999999999996}]},{"osd":9,"last update":"Fri Jul 14 11:59:16 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.91700000000000004}]},{"osd":10,"last update":"Fri Jul 14 11:59:35 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.871},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.84099999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]}]},{"osd":4,"up_from":34,"seq":146028888114,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":974848,"data_stored":486237,"data_compressed":17563,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.52700000000000002,"15min":0.504},"min":{"1min":0.41399999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.53700000000000003,"15min":0.52300000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.80000000000000004,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.48299999999999998}]},{"osd":1,"last update":"Fri Jul 14 11:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.36199999999999999,"15min":0.35099999999999998},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.377,"15min":0.372},"min":{"1min":0.23999999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.57799999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.38800000000000001}]},{"osd":2,"last update":"Fri Jul 14 11:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54300000000000004,"15min":0.53700000000000003},"min":{"1min":0.39100000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.57199999999999995,"15min":0.55000000000000004},"min":{"1min":0.38800000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.58999999999999997}]},{"osd":3,"last update":"Fri Jul 14 11:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.56699999999999995,"15min":0.54900000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85399999999999998,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.57599999999999996,"15min":0.56399999999999995},"min":{"1min":0.47099999999999997,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.75800000000000001,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.80500000000000005}]},{"osd":5,"last update":"Fri Jul 14 11:59:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.58999999999999997,"15min":0.57799999999999996},"min":{"1min":0.41199999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.85399999999999998,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58999999999999997,"15min":0.58599999999999997},"min":{"1min":0.42599999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81699999999999995,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.51000000000000001}]},{"osd":6,"last update":"Fri Jul 14 12:00:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.623,"15min":0.61599999999999999},"min":{"1min":0.46800000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.80800000000000005,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.69199999999999995,"15min":0.68400000000000005},"min":{"1min":0.48299999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.86099999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.55400000000000005,"15min":0.54300000000000004},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.53100000000000003,"15min":0.51600000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.47299999999999998}]},{"osd":8,"last update":"Fri Jul 14 11:59:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.61699999999999999,"15min":0.61499999999999999},"min":{"1min":0.46600000000000003,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.84599999999999997,"5min":0.874,"15min":0.874},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.51800000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.032,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.78800000000000003}]},{"osd":9,"last update":"Fri Jul 14 11:59:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.92800000000000005}]},{"osd":10,"last update":"Fri Jul 14 11:59:37 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.64000000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]}]},{"osd":5,"up_from":39,"seq":167503724590,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":1044480,"data_stored":1077680,"data_compressed":53729,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:59:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60399999999999998,"15min":0.60499999999999998},"min":{"1min":0.36699999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.78600000000000003,"5min":1.018,"15min":1.018},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.59899999999999998,"15min":0.60899999999999999},"min":{"1min":0.34599999999999997,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.81299999999999994,"5min":1.127,"15min":1.127},"last":0.54000000000000004}]},{"osd":1,"last update":"Fri Jul 14 11:59:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.54800000000000004,"15min":0.52600000000000002},"min":{"1min":0.41199999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.875,"5min":1.018,"15min":1.018},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.54800000000000004,"15min":0.53200000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.82299999999999995,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.65200000000000002}]},{"osd":2,"last update":"Fri Jul 14 11:59:37 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.38600000000000001,"15min":0.378},"min":{"1min":0.26800000000000002,"5min":0.22,"15min":0.22},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.41099999999999998,"15min":0.40799999999999997},"min":{"1min":0.309,"5min":0.253,"15min":0.253},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.58599999999999997}]},{"osd":3,"last update":"Fri Jul 14 11:59:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.65400000000000003,"15min":0.64400000000000002},"min":{"1min":0.40000000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.77700000000000002,"5min":3.6640000000000001,"15min":3.6640000000000001},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.65500000000000003,"15min":0.65600000000000003},"min":{"1min":0.42699999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.82399999999999995,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.90900000000000003}]},{"osd":4,"last update":"Fri Jul 14 11:59:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56599999999999995,"15min":0.55200000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.75900000000000001,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57399999999999995,"15min":0.56499999999999995},"min":{"1min":0.39900000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.84199999999999997,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":1.0009999999999999}]},{"osd":6,"last update":"Fri Jul 14 12:00:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.67400000000000004,"15min":0.67000000000000004},"min":{"1min":0.46800000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.91800000000000004,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":1.351},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.69699999999999995,"15min":0.69599999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.97599999999999998}]},{"osd":7,"last update":"Fri Jul 14 11:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.66000000000000003,"15min":0.66200000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.72899999999999998,"15min":0.74299999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.84799999999999998,"5min":3.6440000000000001,"15min":3.6440000000000001},"last":1.0369999999999999}]},{"osd":8,"last update":"Fri Jul 14 11:59:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57699999999999996,"15min":0.57099999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.84999999999999998,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.754},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57299999999999995,"15min":0.57599999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.73699999999999999,"5min":0.874,"15min":0.874},"last":1.0820000000000001}]},{"osd":9,"last update":"Fri Jul 14 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.79000000000000004,"15min":0.78400000000000003},"min":{"1min":0.59499999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":1.389},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.77600000000000002,"15min":0.77000000000000002},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.875}]},{"osd":10,"last update":"Fri Jul 14 11:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":1.1399999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":1.2949999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36166,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36166,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36166,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36166,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36166,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T12:00:16.918 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-14T12:00:17.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:16 smithi093 ceph-mon[128922]: pgmap v295: 129 pgs: 129 active+clean; 583 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 2 op/s 2023-07-14T12:00:17.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:16 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/278934150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T12:00:17.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:16 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/3153423548' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T12:00:17.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:16 smithi093 ceph-mon[128922]: mds.? [v2:172.21.15.17:6838/2976493015,v1:172.21.15.17:6839/2976493015] up:active 2023-07-14T12:00:17.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:16 smithi093 ceph-mon[128922]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-07-14T12:00:17.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:16 smithi093 ceph-mon[128922]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-07-14T12:00:17.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:16 smithi093 ceph-mon[128922]: Cluster is now healthy 2023-07-14T12:00:17.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:16 smithi093 ceph-mon[128922]: fsmap cephfs:4 {0=a=up:active,1=b=up:active,2=i=up:active,3=g=up:active} 8 up:standby 2023-07-14T12:00:17.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:16 smithi093 ceph-mon[128922]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2023-07-14T12:00:17.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:16 smithi093 ceph-mon[128922]: daemon mds.c is now active in filesystem cephfs as rank 4 2023-07-14T12:00:17.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:16 smithi097 ceph-mon[129313]: pgmap v295: 129 pgs: 129 active+clean; 583 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 2 op/s 2023-07-14T12:00:17.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:16 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/278934150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T12:00:17.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:16 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/3153423548' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T12:00:17.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:16 smithi097 ceph-mon[129313]: mds.? [v2:172.21.15.17:6838/2976493015,v1:172.21.15.17:6839/2976493015] up:active 2023-07-14T12:00:17.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:16 smithi097 ceph-mon[129313]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-07-14T12:00:17.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:16 smithi097 ceph-mon[129313]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-07-14T12:00:17.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:16 smithi097 ceph-mon[129313]: Cluster is now healthy 2023-07-14T12:00:17.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:16 smithi097 ceph-mon[129313]: fsmap cephfs:4 {0=a=up:active,1=b=up:active,2=i=up:active,3=g=up:active} 8 up:standby 2023-07-14T12:00:17.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:16 smithi097 ceph-mon[129313]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2023-07-14T12:00:17.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:16 smithi097 ceph-mon[129313]: daemon mds.c is now active in filesystem cephfs as rank 4 2023-07-14T12:00:17.143 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:16 smithi017 ceph-mon[126339]: pgmap v295: 129 pgs: 129 active+clean; 583 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 2 op/s 2023-07-14T12:00:17.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:16 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/278934150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T12:00:17.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:16 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/3153423548' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T12:00:17.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:16 smithi017 ceph-mon[126339]: mds.? [v2:172.21.15.17:6838/2976493015,v1:172.21.15.17:6839/2976493015] up:active 2023-07-14T12:00:17.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:16 smithi017 ceph-mon[126339]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-07-14T12:00:17.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:16 smithi017 ceph-mon[126339]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-07-14T12:00:17.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:16 smithi017 ceph-mon[126339]: Cluster is now healthy 2023-07-14T12:00:17.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:16 smithi017 ceph-mon[126339]: fsmap cephfs:4 {0=a=up:active,1=b=up:active,2=i=up:active,3=g=up:active} 8 up:standby 2023-07-14T12:00:17.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:16 smithi017 ceph-mon[126339]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2023-07-14T12:00:17.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:16 smithi017 ceph-mon[126339]: daemon mds.c is now active in filesystem cephfs as rank 4 2023-07-14T12:00:17.149 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.169+0000 7f72f37ff700 1 -- 172.21.15.17:0/1502849558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72ec100e30 msgr2=0x7f72ec1012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:17.149 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.169+0000 7f72f37ff700 1 --2- 172.21.15.17:0/1502849558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72ec100e30 0x7f72ec1012a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f72dc009990 tx=0x7f72dc009ca0 comp rx=0 tx=0).stop 2023-07-14T12:00:17.149 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.169+0000 7f72f37ff700 1 -- 172.21.15.17:0/1502849558 shutdown_connections 2023-07-14T12:00:17.149 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.169+0000 7f72f37ff700 1 --2- 172.21.15.17:0/1502849558 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f72ec0ffba0 0x7f72ec0fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:17.149 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.169+0000 7f72f37ff700 1 --2- 172.21.15.17:0/1502849558 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72ec1017e0 0x7f72ec108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:17.150 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.169+0000 7f72f37ff700 1 --2- 172.21.15.17:0/1502849558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72ec100e30 0x7f72ec1012a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:17.150 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.169+0000 7f72f37ff700 1 -- 172.21.15.17:0/1502849558 >> 172.21.15.17:0/1502849558 conn(0x7f72ec0fb1c0 msgr2=0x7f72ec0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:17.150 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.170+0000 7f72f37ff700 1 -- 172.21.15.17:0/1502849558 shutdown_connections 2023-07-14T12:00:17.150 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.170+0000 7f72f37ff700 1 -- 172.21.15.17:0/1502849558 wait complete. 2023-07-14T12:00:17.150 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.171+0000 7f72f37ff700 1 Processor -- start 2023-07-14T12:00:17.151 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.171+0000 7f72f37ff700 1 -- start start 2023-07-14T12:00:17.152 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.172+0000 7f72f37ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f72ec0ffba0 0x7f72ec195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:17.152 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.172+0000 7f72f37ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72ec100e30 0x7f72ec195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:17.152 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.172+0000 7f72f37ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72ec1017e0 0x7f72ec19cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:17.152 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.172+0000 7f72f37ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ec19d4f0 con 0x7f72ec100e30 2023-07-14T12:00:17.152 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.172+0000 7f72f37ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ec19d630 con 0x7f72ec0ffba0 2023-07-14T12:00:17.152 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.172+0000 7f72f37ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ec19d7c0 con 0x7f72ec1017e0 2023-07-14T12:00:17.152 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.172+0000 7f72f159b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f72ec0ffba0 0x7f72ec195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:17.153 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.172+0000 7f72f0d9a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72ec100e30 0x7f72ec195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:17.153 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.172+0000 7f72f159b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f72ec0ffba0 0x7f72ec195970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.17:57666/0 (socket says 172.21.15.17:57666) 2023-07-14T12:00:17.153 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.172+0000 7f72f159b700 1 -- 172.21.15.17:0/2612279231 learned_addr learned my addr 172.21.15.17:0/2612279231 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:00:17.153 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.172+0000 7f72f1d9c700 1 --2- 172.21.15.17:0/2612279231 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72ec1017e0 0x7f72ec19cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:17.154 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.173+0000 7f72f159b700 1 -- 172.21.15.17:0/2612279231 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72ec1017e0 msgr2=0x7f72ec19cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:17.154 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.173+0000 7f72f159b700 1 --2- 172.21.15.17:0/2612279231 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72ec1017e0 0x7f72ec19cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:17.154 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.173+0000 7f72f159b700 1 -- 172.21.15.17:0/2612279231 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72ec100e30 msgr2=0x7f72ec195eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:17.154 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.173+0000 7f72f159b700 1 --2- 172.21.15.17:0/2612279231 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72ec100e30 0x7f72ec195eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:17.155 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.173+0000 7f72f159b700 1 -- 172.21.15.17:0/2612279231 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e4009c40 con 0x7f72ec0ffba0 2023-07-14T12:00:17.155 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.173+0000 7f72f0d9a700 1 --2- 172.21.15.17:0/2612279231 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72ec100e30 0x7f72ec195eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T12:00:17.155 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.173+0000 7f72f159b700 1 --2- 172.21.15.17:0/2612279231 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f72ec0ffba0 0x7f72ec195970 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f72e800e9a0 tx=0x7f72e800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:17.155 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.173+0000 7f72e27fc700 1 -- 172.21.15.17:0/2612279231 <== mon.2 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e8009790 con 0x7f72ec0ffba0 2023-07-14T12:00:17.155 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.173+0000 7f72e27fc700 1 -- 172.21.15.17:0/2612279231 <== mon.2 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72e800ce80 con 0x7f72ec0ffba0 2023-07-14T12:00:17.155 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.173+0000 7f72f37ff700 1 -- 172.21.15.17:0/2612279231 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72dc0095f0 con 0x7f72ec0ffba0 2023-07-14T12:00:17.155 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.173+0000 7f72e27fc700 1 -- 172.21.15.17:0/2612279231 <== mon.2 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e8018680 con 0x7f72ec0ffba0 2023-07-14T12:00:17.155 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.173+0000 7f72f37ff700 1 -- 172.21.15.17:0/2612279231 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72ec19de20 con 0x7f72ec0ffba0 2023-07-14T12:00:17.156 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.174+0000 7f72f37ff700 1 -- 172.21.15.17:0/2612279231 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72ec107280 con 0x7f72ec0ffba0 2023-07-14T12:00:17.156 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.177+0000 7f72e27fc700 1 -- 172.21.15.17:0/2612279231 <== mon.2 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f72e8010430 con 0x7f72ec0ffba0 2023-07-14T12:00:17.157 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.178+0000 7f72e27fc700 1 --2- 172.21.15.17:0/2612279231 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f72d80a1150 0x7f72d80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:17.157 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.178+0000 7f72e27fc700 1 -- 172.21.15.17:0/2612279231 <== mon.2 v2:172.21.15.93:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f72e8014070 con 0x7f72ec0ffba0 2023-07-14T12:00:17.158 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.178+0000 7f72f0d9a700 1 --2- 172.21.15.17:0/2612279231 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f72d80a1150 0x7f72d80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:17.158 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.179+0000 7f72f0d9a700 1 --2- 172.21.15.17:0/2612279231 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f72d80a1150 0x7f72d80a3600 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00b2e0 tx=0x7f72dc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:17.159 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.180+0000 7f72e27fc700 1 -- 172.21.15.17:0/2612279231 <== mon.2 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f72e8096600 con 0x7f72ec0ffba0 2023-07-14T12:00:17.366 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.386+0000 7f72f37ff700 1 -- 172.21.15.17:0/2612279231 --> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f72ec060950 con 0x7f72d80a1150 2023-07-14T12:00:17.379 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.399+0000 7f72e27fc700 1 -- 172.21.15.17:0/2612279231 <== mgr.14152 v2:172.21.15.17:6800/4237449858 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+423345 (secure 0 0 0) 0x7f72ec060950 con 0x7f72d80a1150 2023-07-14T12:00:17.383 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T12:00:17.385 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.404+0000 7f72f37ff700 1 -- 172.21.15.17:0/2612279231 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f72d80a1150 msgr2=0x7f72d80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:17.386 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.404+0000 7f72f37ff700 1 --2- 172.21.15.17:0/2612279231 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f72d80a1150 0x7f72d80a3600 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00b2e0 tx=0x7f72dc005fb0 comp rx=0 tx=0).stop 2023-07-14T12:00:17.386 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.404+0000 7f72f37ff700 1 -- 172.21.15.17:0/2612279231 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f72ec0ffba0 msgr2=0x7f72ec195970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:17.386 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.404+0000 7f72f37ff700 1 --2- 172.21.15.17:0/2612279231 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f72ec0ffba0 0x7f72ec195970 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f72e800e9a0 tx=0x7f72e800ecb0 comp rx=0 tx=0).stop 2023-07-14T12:00:17.386 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.404+0000 7f72f37ff700 1 -- 172.21.15.17:0/2612279231 shutdown_connections 2023-07-14T12:00:17.386 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.404+0000 7f72f37ff700 1 --2- 172.21.15.17:0/2612279231 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f72d80a1150 0x7f72d80a3600 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:17.387 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.404+0000 7f72f37ff700 1 --2- 172.21.15.17:0/2612279231 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f72ec0ffba0 0x7f72ec195970 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:17.387 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.404+0000 7f72f37ff700 1 --2- 172.21.15.17:0/2612279231 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72ec1017e0 0x7f72ec19cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:17.387 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.404+0000 7f72f37ff700 1 --2- 172.21.15.17:0/2612279231 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72ec100e30 0x7f72ec195eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:17.387 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.404+0000 7f72f37ff700 1 -- 172.21.15.17:0/2612279231 >> 172.21.15.17:0/2612279231 conn(0x7f72ec0fb1c0 msgr2=0x7f72ec1043c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:17.387 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.404+0000 7f72f37ff700 1 -- 172.21.15.17:0/2612279231 shutdown_connections 2023-07-14T12:00:17.387 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:17.405+0000 7f72f37ff700 1 -- 172.21.15.17:0/2612279231 wait complete. 2023-07-14T12:00:17.387 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-07-14T12:00:17.403 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2023-07-14T12:00:15.732612+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596780,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":255,"num_write_kb":2969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335052,"kb_used_data":11628,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606772,"statfs":{"total":1151940427776,"available":1151597334528,"internally_reserved":0,"allocated":11907072,"data_stored":7620287,"data_compressed":319254,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1961,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.931592"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.813006+0000","last_change":"2023-07-14T11:59:43.468306+0000","last_active":"2023-07-14T12:00:06.813006+0000","last_peered":"2023-07-14T12:00:06.813006+0000","last_clean":"2023-07-14T12:00:06.813006+0000","last_became_active":"2023-07-14T11:59:43.467506+0000","last_became_peered":"2023-07-14T11:59:43.467506+0000","last_unstale":"2023-07-14T12:00:06.813006+0000","last_undegraded":"2023-07-14T12:00:06.813006+0000","last_fullsized":"2023-07-14T12:00:06.813006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:49:26.343044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.814808+0000","last_change":"2023-07-14T11:59:43.462562+0000","last_active":"2023-07-14T12:00:06.814808+0000","last_peered":"2023-07-14T12:00:06.814808+0000","last_clean":"2023-07-14T12:00:06.814808+0000","last_became_active":"2023-07-14T11:59:43.462126+0000","last_became_peered":"2023-07-14T11:59:43.462126+0000","last_unstale":"2023-07-14T12:00:06.814808+0000","last_undegraded":"2023-07-14T12:00:06.814808+0000","last_fullsized":"2023-07-14T12:00:06.814808+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:20:10.445422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.813957+0000","last_change":"2023-07-14T11:59:43.470205+0000","last_active":"2023-07-14T12:00:06.813957+0000","last_peered":"2023-07-14T12:00:06.813957+0000","last_clean":"2023-07-14T12:00:06.813957+0000","last_became_active":"2023-07-14T11:59:43.469804+0000","last_became_peered":"2023-07-14T11:59:43.469804+0000","last_unstale":"2023-07-14T12:00:06.813957+0000","last_undegraded":"2023-07-14T12:00:06.813957+0000","last_fullsized":"2023-07-14T12:00:06.813957+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:06:21.532674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.807812+0000","last_change":"2023-07-14T11:59:43.464834+0000","last_active":"2023-07-14T12:00:06.807812+0000","last_peered":"2023-07-14T12:00:06.807812+0000","last_clean":"2023-07-14T12:00:06.807812+0000","last_became_active":"2023-07-14T11:59:43.464252+0000","last_became_peered":"2023-07-14T11:59:43.464252+0000","last_unstale":"2023-07-14T12:00:06.807812+0000","last_undegraded":"2023-07-14T12:00:06.807812+0000","last_fullsized":"2023-07-14T12:00:06.807812+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:30:26.528049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808013+0000","last_change":"2023-07-14T11:59:43.460468+0000","last_active":"2023-07-14T12:00:06.808013+0000","last_peered":"2023-07-14T12:00:06.808013+0000","last_clean":"2023-07-14T12:00:06.808013+0000","last_became_active":"2023-07-14T11:59:43.460050+0000","last_became_peered":"2023-07-14T11:59:43.460050+0000","last_unstale":"2023-07-14T12:00:06.808013+0000","last_undegraded":"2023-07-14T12:00:06.808013+0000","last_fullsized":"2023-07-14T12:00:06.808013+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:43:21.915056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808330+0000","last_change":"2023-07-14T11:59:43.458605+0000","last_active":"2023-07-14T12:00:06.808330+0000","last_peered":"2023-07-14T12:00:06.808330+0000","last_clean":"2023-07-14T12:00:06.808330+0000","last_became_active":"2023-07-14T11:59:43.458233+0000","last_became_peered":"2023-07-14T11:59:43.458233+0000","last_unstale":"2023-07-14T12:00:06.808330+0000","last_undegraded":"2023-07-14T12:00:06.808330+0000","last_fullsized":"2023-07-14T12:00:06.808330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:27:55.968254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.328508+0000","last_change":"2023-07-14T11:59:43.465950+0000","last_active":"2023-07-14T12:00:07.328508+0000","last_peered":"2023-07-14T12:00:07.328508+0000","last_clean":"2023-07-14T12:00:07.328508+0000","last_became_active":"2023-07-14T11:59:43.465652+0000","last_became_peered":"2023-07-14T11:59:43.465652+0000","last_unstale":"2023-07-14T12:00:07.328508+0000","last_undegraded":"2023-07-14T12:00:07.328508+0000","last_fullsized":"2023-07-14T12:00:07.328508+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:48:42.466464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.816094+0000","last_change":"2023-07-14T12:00:06.812609+0000","last_active":"2023-07-14T12:00:06.816094+0000","last_peered":"2023-07-14T12:00:06.816094+0000","last_clean":"2023-07-14T12:00:06.816094+0000","last_became_active":"2023-07-14T12:00:06.812112+0000","last_became_peered":"2023-07-14T12:00:06.812112+0000","last_unstale":"2023-07-14T12:00:06.816094+0000","last_undegraded":"2023-07-14T12:00:06.816094+0000","last_fullsized":"2023-07-14T12:00:06.816094+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:41:31.808111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805908+0000","last_change":"2023-07-14T11:59:43.466351+0000","last_active":"2023-07-14T12:00:06.805908+0000","last_peered":"2023-07-14T12:00:06.805908+0000","last_clean":"2023-07-14T12:00:06.805908+0000","last_became_active":"2023-07-14T11:59:43.465442+0000","last_became_peered":"2023-07-14T11:59:43.465442+0000","last_unstale":"2023-07-14T12:00:06.805908+0000","last_undegraded":"2023-07-14T12:00:06.805908+0000","last_fullsized":"2023-07-14T12:00:06.805908+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:52:04.297945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.809112+0000","last_change":"2023-07-14T11:59:43.457684+0000","last_active":"2023-07-14T12:00:06.809112+0000","last_peered":"2023-07-14T12:00:06.809112+0000","last_clean":"2023-07-14T12:00:06.809112+0000","last_became_active":"2023-07-14T11:59:43.457442+0000","last_became_peered":"2023-07-14T11:59:43.457442+0000","last_unstale":"2023-07-14T12:00:06.809112+0000","last_undegraded":"2023-07-14T12:00:06.809112+0000","last_fullsized":"2023-07-14T12:00:06.809112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:06:41.592243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.328566+0000","last_change":"2023-07-14T11:59:43.463652+0000","last_active":"2023-07-14T12:00:07.328566+0000","last_peered":"2023-07-14T12:00:07.328566+0000","last_clean":"2023-07-14T12:00:07.328566+0000","last_became_active":"2023-07-14T11:59:43.463311+0000","last_became_peered":"2023-07-14T11:59:43.463311+0000","last_unstale":"2023-07-14T12:00:07.328566+0000","last_undegraded":"2023-07-14T12:00:07.328566+0000","last_fullsized":"2023-07-14T12:00:07.328566+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:28:35.177490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808344+0000","last_change":"2023-07-14T11:59:43.464633+0000","last_active":"2023-07-14T12:00:06.808344+0000","last_peered":"2023-07-14T12:00:06.808344+0000","last_clean":"2023-07-14T12:00:06.808344+0000","last_became_active":"2023-07-14T11:59:43.463847+0000","last_became_peered":"2023-07-14T11:59:43.463847+0000","last_unstale":"2023-07-14T12:00:06.808344+0000","last_undegraded":"2023-07-14T12:00:06.808344+0000","last_fullsized":"2023-07-14T12:00:06.808344+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:08:28.236596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808316+0000","last_change":"2023-07-14T11:59:43.465650+0000","last_active":"2023-07-14T12:00:06.808316+0000","last_peered":"2023-07-14T12:00:06.808316+0000","last_clean":"2023-07-14T12:00:06.808316+0000","last_became_active":"2023-07-14T11:59:43.465276+0000","last_became_peered":"2023-07-14T11:59:43.465276+0000","last_unstale":"2023-07-14T12:00:06.808316+0000","last_undegraded":"2023-07-14T12:00:06.808316+0000","last_fullsized":"2023-07-14T12:00:06.808316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:00:25.094505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.812937+0000","last_change":"2023-07-14T11:59:43.466437+0000","last_active":"2023-07-14T12:00:06.812937+0000","last_peered":"2023-07-14T12:00:06.812937+0000","last_clean":"2023-07-14T12:00:06.812937+0000","last_became_active":"2023-07-14T11:59:43.465361+0000","last_became_peered":"2023-07-14T11:59:43.465361+0000","last_unstale":"2023-07-14T12:00:06.812937+0000","last_undegraded":"2023-07-14T12:00:06.812937+0000","last_fullsized":"2023-07-14T12:00:06.812937+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:31:26.002175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.814761+0000","last_change":"2023-07-14T11:59:43.469452+0000","last_active":"2023-07-14T12:00:06.814761+0000","last_peered":"2023-07-14T12:00:06.814761+0000","last_clean":"2023-07-14T12:00:06.814761+0000","last_became_active":"2023-07-14T11:59:43.469112+0000","last_became_peered":"2023-07-14T11:59:43.469112+0000","last_unstale":"2023-07-14T12:00:06.814761+0000","last_undegraded":"2023-07-14T12:00:06.814761+0000","last_fullsized":"2023-07-14T12:00:06.814761+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:25:31.596988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.804882+0000","last_change":"2023-07-14T11:59:43.466746+0000","last_active":"2023-07-14T12:00:06.804882+0000","last_peered":"2023-07-14T12:00:06.804882+0000","last_clean":"2023-07-14T12:00:06.804882+0000","last_became_active":"2023-07-14T11:59:43.466530+0000","last_became_peered":"2023-07-14T11:59:43.466530+0000","last_unstale":"2023-07-14T12:00:06.804882+0000","last_undegraded":"2023-07-14T12:00:06.804882+0000","last_fullsized":"2023-07-14T12:00:06.804882+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:47:15.633490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.812795+0000","last_change":"2023-07-14T11:59:43.468510+0000","last_active":"2023-07-14T12:00:06.812795+0000","last_peered":"2023-07-14T12:00:06.812795+0000","last_clean":"2023-07-14T12:00:06.812795+0000","last_became_active":"2023-07-14T11:59:43.467990+0000","last_became_peered":"2023-07-14T11:59:43.467990+0000","last_unstale":"2023-07-14T12:00:06.812795+0000","last_undegraded":"2023-07-14T12:00:06.812795+0000","last_fullsized":"2023-07-14T12:00:06.812795+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:35:21.496112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805431+0000","last_change":"2023-07-14T11:59:43.465290+0000","last_active":"2023-07-14T12:00:06.805431+0000","last_peered":"2023-07-14T12:00:06.805431+0000","last_clean":"2023-07-14T12:00:06.805431+0000","last_became_active":"2023-07-14T11:59:43.464982+0000","last_became_peered":"2023-07-14T11:59:43.464982+0000","last_unstale":"2023-07-14T12:00:06.805431+0000","last_undegraded":"2023-07-14T12:00:06.805431+0000","last_fullsized":"2023-07-14T12:00:06.805431+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:49:36.157904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.806693+0000","last_change":"2023-07-14T11:59:43.465053+0000","last_active":"2023-07-14T12:00:06.806693+0000","last_peered":"2023-07-14T12:00:06.806693+0000","last_clean":"2023-07-14T12:00:06.806693+0000","last_became_active":"2023-07-14T11:59:43.464759+0000","last_became_peered":"2023-07-14T11:59:43.464759+0000","last_unstale":"2023-07-14T12:00:06.806693+0000","last_undegraded":"2023-07-14T12:00:06.806693+0000","last_fullsized":"2023-07-14T12:00:06.806693+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:41:15.313602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805157+0000","last_change":"2023-07-14T11:59:47.792164+0000","last_active":"2023-07-14T12:00:06.805157+0000","last_peered":"2023-07-14T12:00:06.805157+0000","last_clean":"2023-07-14T12:00:06.805157+0000","last_became_active":"2023-07-14T11:59:47.791749+0000","last_became_peered":"2023-07-14T11:59:47.791749+0000","last_unstale":"2023-07-14T12:00:06.805157+0000","last_undegraded":"2023-07-14T12:00:06.805157+0000","last_fullsized":"2023-07-14T12:00:06.805157+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:32:07.262881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805207+0000","last_change":"2023-07-14T11:59:42.759256+0000","last_active":"2023-07-14T12:00:06.805207+0000","last_peered":"2023-07-14T12:00:06.805207+0000","last_clean":"2023-07-14T12:00:06.805207+0000","last_became_active":"2023-07-14T11:59:42.758938+0000","last_became_peered":"2023-07-14T11:59:42.758938+0000","last_unstale":"2023-07-14T12:00:06.805207+0000","last_undegraded":"2023-07-14T12:00:06.805207+0000","last_fullsized":"2023-07-14T12:00:06.805207+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:21:38.079963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.811011+0000","last_change":"2023-07-14T11:59:43.467503+0000","last_active":"2023-07-14T12:00:06.811011+0000","last_peered":"2023-07-14T12:00:06.811011+0000","last_clean":"2023-07-14T12:00:06.811011+0000","last_became_active":"2023-07-14T11:59:43.467130+0000","last_became_peered":"2023-07-14T11:59:43.467130+0000","last_unstale":"2023-07-14T12:00:06.811011+0000","last_undegraded":"2023-07-14T12:00:06.811011+0000","last_fullsized":"2023-07-14T12:00:06.811011+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:26:08.037607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.807340+0000","last_change":"2023-07-14T11:59:47.783514+0000","last_active":"2023-07-14T12:00:06.807340+0000","last_peered":"2023-07-14T12:00:06.807340+0000","last_clean":"2023-07-14T12:00:06.807340+0000","last_became_active":"2023-07-14T11:59:47.783115+0000","last_became_peered":"2023-07-14T11:59:47.783115+0000","last_unstale":"2023-07-14T12:00:06.807340+0000","last_undegraded":"2023-07-14T12:00:06.807340+0000","last_fullsized":"2023-07-14T12:00:06.807340+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:19:30.538776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"83'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.811088+0000","last_change":"2023-07-14T11:59:42.768975+0000","last_active":"2023-07-14T12:00:06.811088+0000","last_peered":"2023-07-14T12:00:06.811088+0000","last_clean":"2023-07-14T12:00:06.811088+0000","last_became_active":"2023-07-14T11:59:42.768603+0000","last_became_peered":"2023-07-14T11:59:42.768603+0000","last_unstale":"2023-07-14T12:00:06.811088+0000","last_undegraded":"2023-07-14T12:00:06.811088+0000","last_fullsized":"2023-07-14T12:00:06.811088+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:20:32.308882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.813232+0000","last_change":"2023-07-14T11:59:43.474252+0000","last_active":"2023-07-14T12:00:06.813232+0000","last_peered":"2023-07-14T12:00:06.813232+0000","last_clean":"2023-07-14T12:00:06.813232+0000","last_became_active":"2023-07-14T11:59:43.473955+0000","last_became_peered":"2023-07-14T11:59:43.473955+0000","last_unstale":"2023-07-14T12:00:06.813232+0000","last_undegraded":"2023-07-14T12:00:06.813232+0000","last_fullsized":"2023-07-14T12:00:06.813232+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:52:46.217372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.813893+0000","last_change":"2023-07-14T12:00:06.813893+0000","last_active":"2023-07-14T12:00:06.813893+0000","last_peered":"2023-07-14T12:00:06.813893+0000","last_clean":"2023-07-14T12:00:06.813893+0000","last_became_active":"2023-07-14T12:00:06.813568+0000","last_became_peered":"2023-07-14T12:00:06.813568+0000","last_unstale":"2023-07-14T12:00:06.813893+0000","last_undegraded":"2023-07-14T12:00:06.813893+0000","last_fullsized":"2023-07-14T12:00:06.813893+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:57:58.591410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808860+0000","last_change":"2023-07-14T11:59:42.760791+0000","last_active":"2023-07-14T12:00:06.808860+0000","last_peered":"2023-07-14T12:00:06.808860+0000","last_clean":"2023-07-14T12:00:06.808860+0000","last_became_active":"2023-07-14T11:59:42.760422+0000","last_became_peered":"2023-07-14T11:59:42.760422+0000","last_unstale":"2023-07-14T12:00:06.808860+0000","last_undegraded":"2023-07-14T12:00:06.808860+0000","last_fullsized":"2023-07-14T12:00:06.808860+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:10:21.008350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805235+0000","last_change":"2023-07-14T11:59:43.455829+0000","last_active":"2023-07-14T12:00:06.805235+0000","last_peered":"2023-07-14T12:00:06.805235+0000","last_clean":"2023-07-14T12:00:06.805235+0000","last_became_active":"2023-07-14T11:59:43.455541+0000","last_became_peered":"2023-07-14T11:59:43.455541+0000","last_unstale":"2023-07-14T12:00:06.805235+0000","last_undegraded":"2023-07-14T12:00:06.805235+0000","last_fullsized":"2023-07-14T12:00:06.805235+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:42:43.607658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.807537+0000","last_change":"2023-07-14T11:59:47.783496+0000","last_active":"2023-07-14T12:00:06.807537+0000","last_peered":"2023-07-14T12:00:06.807537+0000","last_clean":"2023-07-14T12:00:06.807537+0000","last_became_active":"2023-07-14T11:59:47.783128+0000","last_became_peered":"2023-07-14T11:59:47.783128+0000","last_unstale":"2023-07-14T12:00:06.807537+0000","last_undegraded":"2023-07-14T12:00:06.807537+0000","last_fullsized":"2023-07-14T12:00:06.807537+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:55:13.905682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.807417+0000","last_change":"2023-07-14T11:59:42.757255+0000","last_active":"2023-07-14T12:00:06.807417+0000","last_peered":"2023-07-14T12:00:06.807417+0000","last_clean":"2023-07-14T12:00:06.807417+0000","last_became_active":"2023-07-14T11:59:42.756882+0000","last_became_peered":"2023-07-14T11:59:42.756882+0000","last_unstale":"2023-07-14T12:00:06.807417+0000","last_undegraded":"2023-07-14T12:00:06.807417+0000","last_fullsized":"2023-07-14T12:00:06.807417+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:22:50.954980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808779+0000","last_change":"2023-07-14T11:59:43.465501+0000","last_active":"2023-07-14T12:00:06.808779+0000","last_peered":"2023-07-14T12:00:06.808779+0000","last_clean":"2023-07-14T12:00:06.808779+0000","last_became_active":"2023-07-14T11:59:43.465218+0000","last_became_peered":"2023-07-14T11:59:43.465218+0000","last_unstale":"2023-07-14T12:00:06.808779+0000","last_undegraded":"2023-07-14T12:00:06.808779+0000","last_fullsized":"2023-07-14T12:00:06.808779+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:02:45.983300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.806670+0000","last_change":"2023-07-14T11:59:47.777816+0000","last_active":"2023-07-14T12:00:06.806670+0000","last_peered":"2023-07-14T12:00:06.806670+0000","last_clean":"2023-07-14T12:00:06.806670+0000","last_became_active":"2023-07-14T11:59:47.777373+0000","last_became_peered":"2023-07-14T11:59:47.777373+0000","last_unstale":"2023-07-14T12:00:06.806670+0000","last_undegraded":"2023-07-14T12:00:06.806670+0000","last_fullsized":"2023-07-14T12:00:06.806670+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:58:17.252326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:13.466245+0000","last_change":"2023-07-14T11:59:42.756362+0000","last_active":"2023-07-14T12:00:13.466245+0000","last_peered":"2023-07-14T12:00:13.466245+0000","last_clean":"2023-07-14T12:00:13.466245+0000","last_became_active":"2023-07-14T11:59:42.756074+0000","last_became_peered":"2023-07-14T11:59:42.756074+0000","last_unstale":"2023-07-14T12:00:13.466245+0000","last_undegraded":"2023-07-14T12:00:13.466245+0000","last_fullsized":"2023-07-14T12:00:13.466245+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:22:56.214797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.812341+0000","last_change":"2023-07-14T12:00:06.812341+0000","last_active":"2023-07-14T12:00:06.812341+0000","last_peered":"2023-07-14T12:00:06.812341+0000","last_clean":"2023-07-14T12:00:06.812341+0000","last_became_active":"2023-07-14T12:00:06.811891+0000","last_became_peered":"2023-07-14T12:00:06.811891+0000","last_unstale":"2023-07-14T12:00:06.812341+0000","last_undegraded":"2023-07-14T12:00:06.812341+0000","last_fullsized":"2023-07-14T12:00:06.812341+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:19:54.770200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.806081+0000","last_change":"2023-07-14T11:59:47.785323+0000","last_active":"2023-07-14T12:00:06.806081+0000","last_peered":"2023-07-14T12:00:06.806081+0000","last_clean":"2023-07-14T12:00:06.806081+0000","last_became_active":"2023-07-14T11:59:47.784892+0000","last_became_peered":"2023-07-14T11:59:47.784892+0000","last_unstale":"2023-07-14T12:00:06.806081+0000","last_undegraded":"2023-07-14T12:00:06.806081+0000","last_fullsized":"2023-07-14T12:00:06.806081+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:48:54.236864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"83'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808194+0000","last_change":"2023-07-14T11:59:42.756364+0000","last_active":"2023-07-14T12:00:06.808194+0000","last_peered":"2023-07-14T12:00:06.808194+0000","last_clean":"2023-07-14T12:00:06.808194+0000","last_became_active":"2023-07-14T11:59:42.756080+0000","last_became_peered":"2023-07-14T11:59:42.756080+0000","last_unstale":"2023-07-14T12:00:06.808194+0000","last_undegraded":"2023-07-14T12:00:06.808194+0000","last_fullsized":"2023-07-14T12:00:06.808194+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:55:28.974638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808819+0000","last_change":"2023-07-14T11:59:43.470331+0000","last_active":"2023-07-14T12:00:06.808819+0000","last_peered":"2023-07-14T12:00:06.808819+0000","last_clean":"2023-07-14T12:00:06.808819+0000","last_became_active":"2023-07-14T11:59:43.469806+0000","last_became_peered":"2023-07-14T11:59:43.469806+0000","last_unstale":"2023-07-14T12:00:06.808819+0000","last_undegraded":"2023-07-14T12:00:06.808819+0000","last_fullsized":"2023-07-14T12:00:06.808819+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:31:39.648120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.812748+0000","last_change":"2023-07-14T11:59:47.784083+0000","last_active":"2023-07-14T12:00:06.812748+0000","last_peered":"2023-07-14T12:00:06.812748+0000","last_clean":"2023-07-14T12:00:06.812748+0000","last_became_active":"2023-07-14T11:59:47.783624+0000","last_became_peered":"2023-07-14T11:59:47.783624+0000","last_unstale":"2023-07-14T12:00:06.812748+0000","last_undegraded":"2023-07-14T12:00:06.812748+0000","last_fullsized":"2023-07-14T12:00:06.812748+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:19:59.493479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.809448+0000","last_change":"2023-07-14T11:59:42.764869+0000","last_active":"2023-07-14T12:00:06.809448+0000","last_peered":"2023-07-14T12:00:06.809448+0000","last_clean":"2023-07-14T12:00:06.809448+0000","last_became_active":"2023-07-14T11:59:42.764514+0000","last_became_peered":"2023-07-14T11:59:42.764514+0000","last_unstale":"2023-07-14T12:00:06.809448+0000","last_undegraded":"2023-07-14T12:00:06.809448+0000","last_fullsized":"2023-07-14T12:00:06.809448+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:14:29.041985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.809908+0000","last_change":"2023-07-14T11:59:43.458165+0000","last_active":"2023-07-14T12:00:06.809908+0000","last_peered":"2023-07-14T12:00:06.809908+0000","last_clean":"2023-07-14T12:00:06.809908+0000","last_became_active":"2023-07-14T11:59:43.457829+0000","last_became_peered":"2023-07-14T11:59:43.457829+0000","last_unstale":"2023-07-14T12:00:06.809908+0000","last_undegraded":"2023-07-14T12:00:06.809908+0000","last_fullsized":"2023-07-14T12:00:06.809908+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:43:38.283487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.814633+0000","last_change":"2023-07-14T11:59:47.782676+0000","last_active":"2023-07-14T12:00:06.814633+0000","last_peered":"2023-07-14T12:00:06.814633+0000","last_clean":"2023-07-14T12:00:06.814633+0000","last_became_active":"2023-07-14T11:59:47.782283+0000","last_became_peered":"2023-07-14T11:59:47.782283+0000","last_unstale":"2023-07-14T12:00:06.814633+0000","last_undegraded":"2023-07-14T12:00:06.814633+0000","last_fullsized":"2023-07-14T12:00:06.814633+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:50:29.299244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.809415+0000","last_change":"2023-07-14T11:59:42.756701+0000","last_active":"2023-07-14T12:00:06.809415+0000","last_peered":"2023-07-14T12:00:06.809415+0000","last_clean":"2023-07-14T12:00:06.809415+0000","last_became_active":"2023-07-14T11:59:42.756452+0000","last_became_peered":"2023-07-14T11:59:42.756452+0000","last_unstale":"2023-07-14T12:00:06.809415+0000","last_undegraded":"2023-07-14T12:00:06.809415+0000","last_fullsized":"2023-07-14T12:00:06.809415+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:56:37.979449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.804650+0000","last_change":"2023-07-14T11:59:43.457029+0000","last_active":"2023-07-14T12:00:06.804650+0000","last_peered":"2023-07-14T12:00:06.804650+0000","last_clean":"2023-07-14T12:00:06.804650+0000","last_became_active":"2023-07-14T11:59:43.456710+0000","last_became_peered":"2023-07-14T11:59:43.456710+0000","last_unstale":"2023-07-14T12:00:06.804650+0000","last_undegraded":"2023-07-14T12:00:06.804650+0000","last_fullsized":"2023-07-14T12:00:06.804650+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:17:39.969654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.811649+0000","last_change":"2023-07-14T11:59:47.785732+0000","last_active":"2023-07-14T12:00:06.811649+0000","last_peered":"2023-07-14T12:00:06.811649+0000","last_clean":"2023-07-14T12:00:06.811649+0000","last_became_active":"2023-07-14T11:59:47.785289+0000","last_became_peered":"2023-07-14T11:59:47.785289+0000","last_unstale":"2023-07-14T12:00:06.811649+0000","last_undegraded":"2023-07-14T12:00:06.811649+0000","last_fullsized":"2023-07-14T12:00:06.811649+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:58:47.873312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:13.463147+0000","last_change":"2023-07-14T11:59:42.760013+0000","last_active":"2023-07-14T12:00:13.463147+0000","last_peered":"2023-07-14T12:00:13.463147+0000","last_clean":"2023-07-14T12:00:13.463147+0000","last_became_active":"2023-07-14T11:59:42.759673+0000","last_became_peered":"2023-07-14T11:59:42.759673+0000","last_unstale":"2023-07-14T12:00:13.463147+0000","last_undegraded":"2023-07-14T12:00:13.463147+0000","last_fullsized":"2023-07-14T12:00:13.463147+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:31:23.656642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805922+0000","last_change":"2023-07-14T11:59:43.455418+0000","last_active":"2023-07-14T12:00:06.805922+0000","last_peered":"2023-07-14T12:00:06.805922+0000","last_clean":"2023-07-14T12:00:06.805922+0000","last_became_active":"2023-07-14T11:59:43.454963+0000","last_became_peered":"2023-07-14T11:59:43.454963+0000","last_unstale":"2023-07-14T12:00:06.805922+0000","last_undegraded":"2023-07-14T12:00:06.805922+0000","last_fullsized":"2023-07-14T12:00:06.805922+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:35:27.262942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805369+0000","last_change":"2023-07-14T11:59:47.784330+0000","last_active":"2023-07-14T12:00:06.805369+0000","last_peered":"2023-07-14T12:00:06.805369+0000","last_clean":"2023-07-14T12:00:06.805369+0000","last_became_active":"2023-07-14T11:59:47.783640+0000","last_became_peered":"2023-07-14T11:59:47.783640+0000","last_unstale":"2023-07-14T12:00:06.805369+0000","last_undegraded":"2023-07-14T12:00:06.805369+0000","last_fullsized":"2023-07-14T12:00:06.805369+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:23:46.613595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:13.465788+0000","last_change":"2023-07-14T11:59:42.758856+0000","last_active":"2023-07-14T12:00:13.465788+0000","last_peered":"2023-07-14T12:00:13.465788+0000","last_clean":"2023-07-14T12:00:13.465788+0000","last_became_active":"2023-07-14T11:59:42.758571+0000","last_became_peered":"2023-07-14T11:59:42.758571+0000","last_unstale":"2023-07-14T12:00:13.465788+0000","last_undegraded":"2023-07-14T12:00:13.465788+0000","last_fullsized":"2023-07-14T12:00:13.465788+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:22:26.607862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.814254+0000","last_change":"2023-07-14T11:59:43.466302+0000","last_active":"2023-07-14T12:00:06.814254+0000","last_peered":"2023-07-14T12:00:06.814254+0000","last_clean":"2023-07-14T12:00:06.814254+0000","last_became_active":"2023-07-14T11:59:43.465958+0000","last_became_peered":"2023-07-14T11:59:43.465958+0000","last_unstale":"2023-07-14T12:00:06.814254+0000","last_undegraded":"2023-07-14T12:00:06.814254+0000","last_fullsized":"2023-07-14T12:00:06.814254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:09:00.018902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.813066+0000","last_change":"2023-07-14T12:00:06.813066+0000","last_active":"2023-07-14T12:00:06.813066+0000","last_peered":"2023-07-14T12:00:06.813066+0000","last_clean":"2023-07-14T12:00:06.813066+0000","last_became_active":"2023-07-14T12:00:06.812460+0000","last_became_peered":"2023-07-14T12:00:06.812460+0000","last_unstale":"2023-07-14T12:00:06.813066+0000","last_undegraded":"2023-07-14T12:00:06.813066+0000","last_fullsized":"2023-07-14T12:00:06.813066+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:47:40.893162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808691+0000","last_change":"2023-07-14T11:59:42.765777+0000","last_active":"2023-07-14T12:00:06.808691+0000","last_peered":"2023-07-14T12:00:06.808691+0000","last_clean":"2023-07-14T12:00:06.808691+0000","last_became_active":"2023-07-14T11:59:42.765536+0000","last_became_peered":"2023-07-14T11:59:42.765536+0000","last_unstale":"2023-07-14T12:00:06.808691+0000","last_undegraded":"2023-07-14T12:00:06.808691+0000","last_fullsized":"2023-07-14T12:00:06.808691+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:27:05.053926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805426+0000","last_change":"2023-07-14T11:59:43.463665+0000","last_active":"2023-07-14T12:00:06.805426+0000","last_peered":"2023-07-14T12:00:06.805426+0000","last_clean":"2023-07-14T12:00:06.805426+0000","last_became_active":"2023-07-14T11:59:43.463368+0000","last_became_peered":"2023-07-14T11:59:43.463368+0000","last_unstale":"2023-07-14T12:00:06.805426+0000","last_undegraded":"2023-07-14T12:00:06.805426+0000","last_fullsized":"2023-07-14T12:00:06.805426+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:49:21.359682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.804608+0000","last_change":"2023-07-14T11:59:47.780706+0000","last_active":"2023-07-14T12:00:06.804608+0000","last_peered":"2023-07-14T12:00:06.804608+0000","last_clean":"2023-07-14T12:00:06.804608+0000","last_became_active":"2023-07-14T11:59:47.780207+0000","last_became_peered":"2023-07-14T11:59:47.780207+0000","last_unstale":"2023-07-14T12:00:06.804608+0000","last_undegraded":"2023-07-14T12:00:06.804608+0000","last_fullsized":"2023-07-14T12:00:06.804608+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:45:25.431762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'12","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:13.463278+0000","last_change":"2023-07-14T11:59:42.757238+0000","last_active":"2023-07-14T12:00:13.463278+0000","last_peered":"2023-07-14T12:00:13.463278+0000","last_clean":"2023-07-14T12:00:13.463278+0000","last_became_active":"2023-07-14T11:59:42.756887+0000","last_became_peered":"2023-07-14T11:59:42.756887+0000","last_unstale":"2023-07-14T12:00:13.463278+0000","last_undegraded":"2023-07-14T12:00:13.463278+0000","last_fullsized":"2023-07-14T12:00:13.463278+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:09:27.857990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3794,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808529+0000","last_change":"2023-07-14T11:59:43.460741+0000","last_active":"2023-07-14T12:00:06.808529+0000","last_peered":"2023-07-14T12:00:06.808529+0000","last_clean":"2023-07-14T12:00:06.808529+0000","last_became_active":"2023-07-14T11:59:43.460423+0000","last_became_peered":"2023-07-14T11:59:43.460423+0000","last_unstale":"2023-07-14T12:00:06.808529+0000","last_undegraded":"2023-07-14T12:00:06.808529+0000","last_fullsized":"2023-07-14T12:00:06.808529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:26:56.645695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.809254+0000","last_change":"2023-07-14T11:59:47.781321+0000","last_active":"2023-07-14T12:00:06.809254+0000","last_peered":"2023-07-14T12:00:06.809254+0000","last_clean":"2023-07-14T12:00:06.809254+0000","last_became_active":"2023-07-14T11:59:47.780941+0000","last_became_peered":"2023-07-14T11:59:47.780941+0000","last_unstale":"2023-07-14T12:00:06.809254+0000","last_undegraded":"2023-07-14T12:00:06.809254+0000","last_fullsized":"2023-07-14T12:00:06.809254+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:25:26.671430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.806651+0000","last_change":"2023-07-14T11:59:42.759403+0000","last_active":"2023-07-14T12:00:06.806651+0000","last_peered":"2023-07-14T12:00:06.806651+0000","last_clean":"2023-07-14T12:00:06.806651+0000","last_became_active":"2023-07-14T11:59:42.759036+0000","last_became_peered":"2023-07-14T11:59:42.759036+0000","last_unstale":"2023-07-14T12:00:06.806651+0000","last_undegraded":"2023-07-14T12:00:06.806651+0000","last_fullsized":"2023-07-14T12:00:06.806651+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:45:13.233768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808124+0000","last_change":"2023-07-14T11:59:43.457644+0000","last_active":"2023-07-14T12:00:06.808124+0000","last_peered":"2023-07-14T12:00:06.808124+0000","last_clean":"2023-07-14T12:00:06.808124+0000","last_became_active":"2023-07-14T11:59:43.457363+0000","last_became_peered":"2023-07-14T11:59:43.457363+0000","last_unstale":"2023-07-14T12:00:06.808124+0000","last_undegraded":"2023-07-14T12:00:06.808124+0000","last_fullsized":"2023-07-14T12:00:06.808124+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:12:17.222392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.812650+0000","last_change":"2023-07-14T11:59:47.789684+0000","last_active":"2023-07-14T12:00:06.812650+0000","last_peered":"2023-07-14T12:00:06.812650+0000","last_clean":"2023-07-14T12:00:06.812650+0000","last_became_active":"2023-07-14T11:59:47.789226+0000","last_became_peered":"2023-07-14T11:59:47.789226+0000","last_unstale":"2023-07-14T12:00:06.812650+0000","last_undegraded":"2023-07-14T12:00:06.812650+0000","last_fullsized":"2023-07-14T12:00:06.812650+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:56:58.961737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:13.466046+0000","last_change":"2023-07-14T11:59:42.751497+0000","last_active":"2023-07-14T12:00:13.466046+0000","last_peered":"2023-07-14T12:00:13.466046+0000","last_clean":"2023-07-14T12:00:13.466046+0000","last_became_active":"2023-07-14T11:59:42.751217+0000","last_became_peered":"2023-07-14T11:59:42.751217+0000","last_unstale":"2023-07-14T12:00:13.466046+0000","last_undegraded":"2023-07-14T12:00:13.466046+0000","last_fullsized":"2023-07-14T12:00:13.466046+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:53:00.140364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.328171+0000","last_change":"2023-07-14T11:59:43.462359+0000","last_active":"2023-07-14T12:00:07.328171+0000","last_peered":"2023-07-14T12:00:07.328171+0000","last_clean":"2023-07-14T12:00:07.328171+0000","last_became_active":"2023-07-14T11:59:43.461781+0000","last_became_peered":"2023-07-14T11:59:43.461781+0000","last_unstale":"2023-07-14T12:00:07.328171+0000","last_undegraded":"2023-07-14T12:00:07.328171+0000","last_fullsized":"2023-07-14T12:00:07.328171+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:10:22.527860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805028+0000","last_change":"2023-07-14T11:59:47.790173+0000","last_active":"2023-07-14T12:00:06.805028+0000","last_peered":"2023-07-14T12:00:06.805028+0000","last_clean":"2023-07-14T12:00:06.805028+0000","last_became_active":"2023-07-14T11:59:47.789418+0000","last_became_peered":"2023-07-14T11:59:47.789418+0000","last_unstale":"2023-07-14T12:00:06.805028+0000","last_undegraded":"2023-07-14T12:00:06.805028+0000","last_fullsized":"2023-07-14T12:00:06.805028+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:54:21.435699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:14.477363+0000","last_change":"2023-07-14T11:59:42.753068+0000","last_active":"2023-07-14T12:00:14.477363+0000","last_peered":"2023-07-14T12:00:14.477363+0000","last_clean":"2023-07-14T12:00:14.477363+0000","last_became_active":"2023-07-14T11:59:42.752794+0000","last_became_peered":"2023-07-14T11:59:42.752794+0000","last_unstale":"2023-07-14T12:00:14.477363+0000","last_undegraded":"2023-07-14T12:00:14.477363+0000","last_fullsized":"2023-07-14T12:00:14.477363+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:31:01.258740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.328012+0000","last_change":"2023-07-14T11:59:43.468655+0000","last_active":"2023-07-14T12:00:07.328012+0000","last_peered":"2023-07-14T12:00:07.328012+0000","last_clean":"2023-07-14T12:00:07.328012+0000","last_became_active":"2023-07-14T11:59:43.468361+0000","last_became_peered":"2023-07-14T11:59:43.468361+0000","last_unstale":"2023-07-14T12:00:07.328012+0000","last_undegraded":"2023-07-14T12:00:07.328012+0000","last_fullsized":"2023-07-14T12:00:07.328012+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:02:47.181093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808379+0000","last_change":"2023-07-14T11:59:47.783644+0000","last_active":"2023-07-14T12:00:06.808379+0000","last_peered":"2023-07-14T12:00:06.808379+0000","last_clean":"2023-07-14T12:00:06.808379+0000","last_became_active":"2023-07-14T11:59:47.783229+0000","last_became_peered":"2023-07-14T11:59:47.783229+0000","last_unstale":"2023-07-14T12:00:06.808379+0000","last_undegraded":"2023-07-14T12:00:06.808379+0000","last_fullsized":"2023-07-14T12:00:06.808379+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:15:35.788258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"83'8","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808245+0000","last_change":"2023-07-14T11:59:42.768482+0000","last_active":"2023-07-14T12:00:06.808245+0000","last_peered":"2023-07-14T12:00:06.808245+0000","last_clean":"2023-07-14T12:00:06.808245+0000","last_became_active":"2023-07-14T11:59:42.768201+0000","last_became_peered":"2023-07-14T11:59:42.768201+0000","last_unstale":"2023-07-14T12:00:06.808245+0000","last_undegraded":"2023-07-14T12:00:06.808245+0000","last_fullsized":"2023-07-14T12:00:06.808245+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:39:29.438053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808385+0000","last_change":"2023-07-14T11:59:43.469651+0000","last_active":"2023-07-14T12:00:06.808385+0000","last_peered":"2023-07-14T12:00:06.808385+0000","last_clean":"2023-07-14T12:00:06.808385+0000","last_became_active":"2023-07-14T11:59:43.465848+0000","last_became_peered":"2023-07-14T11:59:43.465848+0000","last_unstale":"2023-07-14T12:00:06.808385+0000","last_undegraded":"2023-07-14T12:00:06.808385+0000","last_fullsized":"2023-07-14T12:00:06.808385+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:17:18.313257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.817649+0000","last_change":"2023-07-14T12:00:06.817110+0000","last_active":"2023-07-14T12:00:06.817649+0000","last_peered":"2023-07-14T12:00:06.817649+0000","last_clean":"2023-07-14T12:00:06.817649+0000","last_became_active":"2023-07-14T12:00:06.816587+0000","last_became_peered":"2023-07-14T12:00:06.816587+0000","last_unstale":"2023-07-14T12:00:06.817649+0000","last_undegraded":"2023-07-14T12:00:06.817649+0000","last_fullsized":"2023-07-14T12:00:06.817649+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:04:52.873643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,0,3,1],"acting":[10,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.f","version":"83'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.807651+0000","last_change":"2023-07-14T11:59:42.758106+0000","last_active":"2023-07-14T12:00:06.807651+0000","last_peered":"2023-07-14T12:00:06.807651+0000","last_clean":"2023-07-14T12:00:06.807651+0000","last_became_active":"2023-07-14T11:59:42.757840+0000","last_became_peered":"2023-07-14T11:59:42.757840+0000","last_unstale":"2023-07-14T12:00:06.807651+0000","last_undegraded":"2023-07-14T12:00:06.807651+0000","last_fullsized":"2023-07-14T12:00:06.807651+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:12:46.164720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.812652+0000","last_change":"2023-07-14T11:59:43.467800+0000","last_active":"2023-07-14T12:00:06.812652+0000","last_peered":"2023-07-14T12:00:06.812652+0000","last_clean":"2023-07-14T12:00:06.812652+0000","last_became_active":"2023-07-14T11:59:43.467448+0000","last_became_peered":"2023-07-14T11:59:43.467448+0000","last_unstale":"2023-07-14T12:00:06.812652+0000","last_undegraded":"2023-07-14T12:00:06.812652+0000","last_fullsized":"2023-07-14T12:00:06.812652+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:34:25.781268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.807574+0000","last_change":"2023-07-14T11:59:47.915884+0000","last_active":"2023-07-14T12:00:06.807574+0000","last_peered":"2023-07-14T12:00:06.807574+0000","last_clean":"2023-07-14T12:00:06.807574+0000","last_became_active":"2023-07-14T11:59:47.915449+0000","last_became_peered":"2023-07-14T11:59:47.915449+0000","last_unstale":"2023-07-14T12:00:06.807574+0000","last_undegraded":"2023-07-14T12:00:06.807574+0000","last_fullsized":"2023-07-14T12:00:06.807574+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:41:55.813157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.804686+0000","last_change":"2023-07-14T11:59:42.771581+0000","last_active":"2023-07-14T12:00:06.804686+0000","last_peered":"2023-07-14T12:00:06.804686+0000","last_clean":"2023-07-14T12:00:06.804686+0000","last_became_active":"2023-07-14T11:59:42.771228+0000","last_became_peered":"2023-07-14T11:59:42.771228+0000","last_unstale":"2023-07-14T12:00:06.804686+0000","last_undegraded":"2023-07-14T12:00:06.804686+0000","last_fullsized":"2023-07-14T12:00:06.804686+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:32:35.711544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808751+0000","last_change":"2023-07-14T11:59:43.460987+0000","last_active":"2023-07-14T12:00:06.808751+0000","last_peered":"2023-07-14T12:00:06.808751+0000","last_clean":"2023-07-14T12:00:06.808751+0000","last_became_active":"2023-07-14T11:59:43.460604+0000","last_became_peered":"2023-07-14T11:59:43.460604+0000","last_unstale":"2023-07-14T12:00:06.808751+0000","last_undegraded":"2023-07-14T12:00:06.808751+0000","last_fullsized":"2023-07-14T12:00:06.808751+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:00:15.767598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808575+0000","last_change":"2023-07-14T11:59:47.788604+0000","last_active":"2023-07-14T12:00:06.808575+0000","last_peered":"2023-07-14T12:00:06.808575+0000","last_clean":"2023-07-14T12:00:06.808575+0000","last_became_active":"2023-07-14T11:59:47.787854+0000","last_became_peered":"2023-07-14T11:59:47.787854+0000","last_unstale":"2023-07-14T12:00:06.808575+0000","last_undegraded":"2023-07-14T12:00:06.808575+0000","last_fullsized":"2023-07-14T12:00:06.808575+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:19:18.510764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.328114+0000","last_change":"2023-07-14T11:59:42.768985+0000","last_active":"2023-07-14T12:00:07.328114+0000","last_peered":"2023-07-14T12:00:07.328114+0000","last_clean":"2023-07-14T12:00:07.328114+0000","last_became_active":"2023-07-14T11:59:42.768688+0000","last_became_peered":"2023-07-14T11:59:42.768688+0000","last_unstale":"2023-07-14T12:00:07.328114+0000","last_undegraded":"2023-07-14T12:00:07.328114+0000","last_fullsized":"2023-07-14T12:00:07.328114+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:20:55.661084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.807718+0000","last_change":"2023-07-14T11:59:43.465529+0000","last_active":"2023-07-14T12:00:06.807718+0000","last_peered":"2023-07-14T12:00:06.807718+0000","last_clean":"2023-07-14T12:00:06.807718+0000","last_became_active":"2023-07-14T11:59:43.465206+0000","last_became_peered":"2023-07-14T11:59:43.465206+0000","last_unstale":"2023-07-14T12:00:06.807718+0000","last_undegraded":"2023-07-14T12:00:06.807718+0000","last_fullsized":"2023-07-14T12:00:06.807718+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:52:33.760607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808309+0000","last_change":"2023-07-14T11:59:47.915842+0000","last_active":"2023-07-14T12:00:06.808309+0000","last_peered":"2023-07-14T12:00:06.808309+0000","last_clean":"2023-07-14T12:00:06.808309+0000","last_became_active":"2023-07-14T11:59:47.915304+0000","last_became_peered":"2023-07-14T11:59:47.915304+0000","last_unstale":"2023-07-14T12:00:06.808309+0000","last_undegraded":"2023-07-14T12:00:06.808309+0000","last_fullsized":"2023-07-14T12:00:06.808309+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:26:43.397241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.809764+0000","last_change":"2023-07-14T11:59:43.472296+0000","last_active":"2023-07-14T12:00:06.809764+0000","last_peered":"2023-07-14T12:00:06.809764+0000","last_clean":"2023-07-14T12:00:06.809764+0000","last_became_active":"2023-07-14T11:59:43.470569+0000","last_became_peered":"2023-07-14T11:59:43.470569+0000","last_unstale":"2023-07-14T12:00:06.809764+0000","last_undegraded":"2023-07-14T12:00:06.809764+0000","last_fullsized":"2023-07-14T12:00:06.809764+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:06:23.546910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.809241+0000","last_change":"2023-07-14T11:59:42.769225+0000","last_active":"2023-07-14T12:00:06.809241+0000","last_peered":"2023-07-14T12:00:06.809241+0000","last_clean":"2023-07-14T12:00:06.809241+0000","last_became_active":"2023-07-14T11:59:42.768877+0000","last_became_peered":"2023-07-14T11:59:42.768877+0000","last_unstale":"2023-07-14T12:00:06.809241+0000","last_undegraded":"2023-07-14T12:00:06.809241+0000","last_fullsized":"2023-07-14T12:00:06.809241+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:19:23.782648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.328375+0000","last_change":"2023-07-14T11:59:47.792665+0000","last_active":"2023-07-14T12:00:07.328375+0000","last_peered":"2023-07-14T12:00:07.328375+0000","last_clean":"2023-07-14T12:00:07.328375+0000","last_became_active":"2023-07-14T11:59:47.792308+0000","last_became_peered":"2023-07-14T11:59:47.792308+0000","last_unstale":"2023-07-14T12:00:07.328375+0000","last_undegraded":"2023-07-14T12:00:07.328375+0000","last_fullsized":"2023-07-14T12:00:07.328375+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:18:05.628049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.810583+0000","last_change":"2023-07-14T11:59:43.461103+0000","last_active":"2023-07-14T12:00:06.810583+0000","last_peered":"2023-07-14T12:00:06.810583+0000","last_clean":"2023-07-14T12:00:06.810583+0000","last_became_active":"2023-07-14T11:59:43.460756+0000","last_became_peered":"2023-07-14T11:59:43.460756+0000","last_unstale":"2023-07-14T12:00:06.810583+0000","last_undegraded":"2023-07-14T12:00:06.810583+0000","last_fullsized":"2023-07-14T12:00:06.810583+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:05:41.752001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:13.465872+0000","last_change":"2023-07-14T11:59:42.752680+0000","last_active":"2023-07-14T12:00:13.465872+0000","last_peered":"2023-07-14T12:00:13.465872+0000","last_clean":"2023-07-14T12:00:13.465872+0000","last_became_active":"2023-07-14T11:59:42.752305+0000","last_became_peered":"2023-07-14T11:59:42.752305+0000","last_unstale":"2023-07-14T12:00:13.465872+0000","last_undegraded":"2023-07-14T12:00:13.465872+0000","last_fullsized":"2023-07-14T12:00:13.465872+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:01:57.359059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":353,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.327820+0000","last_change":"2023-07-14T11:58:37.124871+0000","last_active":"2023-07-14T12:00:07.327820+0000","last_peered":"2023-07-14T12:00:07.327820+0000","last_clean":"2023-07-14T12:00:07.327820+0000","last_became_active":"2023-07-14T11:58:37.124398+0000","last_became_peered":"2023-07-14T11:58:37.124398+0000","last_unstale":"2023-07-14T12:00:07.327820+0000","last_undegraded":"2023-07-14T12:00:07.327820+0000","last_fullsized":"2023-07-14T12:00:07.327820+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:55:28.287160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:55:28.287160+0000","last_clean_scrub_stamp":"2023-07-14T11:55:28.287160+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:00:38.717405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.815687+0000","last_change":"2023-07-14T12:00:06.815687+0000","last_active":"2023-07-14T12:00:06.815687+0000","last_peered":"2023-07-14T12:00:06.815687+0000","last_clean":"2023-07-14T12:00:06.815687+0000","last_became_active":"2023-07-14T12:00:06.815281+0000","last_became_peered":"2023-07-14T12:00:06.815281+0000","last_unstale":"2023-07-14T12:00:06.815687+0000","last_undegraded":"2023-07-14T12:00:06.815687+0000","last_fullsized":"2023-07-14T12:00:06.815687+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:04:42.469562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.813952+0000","last_change":"2023-07-14T12:00:06.813952+0000","last_active":"2023-07-14T12:00:06.813952+0000","last_peered":"2023-07-14T12:00:06.813952+0000","last_clean":"2023-07-14T12:00:06.813952+0000","last_became_active":"2023-07-14T12:00:06.813581+0000","last_became_peered":"2023-07-14T12:00:06.813581+0000","last_unstale":"2023-07-14T12:00:06.813952+0000","last_undegraded":"2023-07-14T12:00:06.813952+0000","last_fullsized":"2023-07-14T12:00:06.813952+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:12:06.783097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.327974+0000","last_change":"2023-07-14T11:59:42.757400+0000","last_active":"2023-07-14T12:00:07.327974+0000","last_peered":"2023-07-14T12:00:07.327974+0000","last_clean":"2023-07-14T12:00:07.327974+0000","last_became_active":"2023-07-14T11:59:42.757047+0000","last_became_peered":"2023-07-14T11:59:42.757047+0000","last_unstale":"2023-07-14T12:00:07.327974+0000","last_undegraded":"2023-07-14T12:00:07.327974+0000","last_fullsized":"2023-07-14T12:00:07.327974+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:42:39.292073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.811396+0000","last_change":"2023-07-14T11:59:47.788486+0000","last_active":"2023-07-14T12:00:06.811396+0000","last_peered":"2023-07-14T12:00:06.811396+0000","last_clean":"2023-07-14T12:00:06.811396+0000","last_became_active":"2023-07-14T11:59:47.787995+0000","last_became_peered":"2023-07-14T11:59:47.787995+0000","last_unstale":"2023-07-14T12:00:06.811396+0000","last_undegraded":"2023-07-14T12:00:06.811396+0000","last_fullsized":"2023-07-14T12:00:06.811396+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:01:52.099573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808657+0000","last_change":"2023-07-14T11:59:42.765810+0000","last_active":"2023-07-14T12:00:06.808657+0000","last_peered":"2023-07-14T12:00:06.808657+0000","last_clean":"2023-07-14T12:00:06.808657+0000","last_became_active":"2023-07-14T11:59:42.765528+0000","last_became_peered":"2023-07-14T11:59:42.765528+0000","last_unstale":"2023-07-14T12:00:06.808657+0000","last_undegraded":"2023-07-14T12:00:06.808657+0000","last_fullsized":"2023-07-14T12:00:06.808657+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:18:02.321077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808724+0000","last_change":"2023-07-14T11:59:43.463763+0000","last_active":"2023-07-14T12:00:06.808724+0000","last_peered":"2023-07-14T12:00:06.808724+0000","last_clean":"2023-07-14T12:00:06.808724+0000","last_became_active":"2023-07-14T11:59:43.463411+0000","last_became_peered":"2023-07-14T11:59:43.463411+0000","last_unstale":"2023-07-14T12:00:06.808724+0000","last_undegraded":"2023-07-14T12:00:06.808724+0000","last_fullsized":"2023-07-14T12:00:06.808724+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:57:24.209856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.809720+0000","last_change":"2023-07-14T11:59:47.916405+0000","last_active":"2023-07-14T12:00:06.809720+0000","last_peered":"2023-07-14T12:00:06.809720+0000","last_clean":"2023-07-14T12:00:06.809720+0000","last_became_active":"2023-07-14T11:59:47.915892+0000","last_became_peered":"2023-07-14T11:59:47.915892+0000","last_unstale":"2023-07-14T12:00:06.809720+0000","last_undegraded":"2023-07-14T12:00:06.809720+0000","last_fullsized":"2023-07-14T12:00:06.809720+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:58:28.977188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"84'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805495+0000","last_change":"2023-07-14T11:59:42.754848+0000","last_active":"2023-07-14T12:00:06.805495+0000","last_peered":"2023-07-14T12:00:06.805495+0000","last_clean":"2023-07-14T12:00:06.805495+0000","last_became_active":"2023-07-14T11:59:42.754522+0000","last_became_peered":"2023-07-14T11:59:42.754522+0000","last_unstale":"2023-07-14T12:00:06.805495+0000","last_undegraded":"2023-07-14T12:00:06.805495+0000","last_fullsized":"2023-07-14T12:00:06.805495+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:49:48.659717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.815209+0000","last_change":"2023-07-14T11:59:43.458954+0000","last_active":"2023-07-14T12:00:06.815209+0000","last_peered":"2023-07-14T12:00:06.815209+0000","last_clean":"2023-07-14T12:00:06.815209+0000","last_became_active":"2023-07-14T11:59:43.458631+0000","last_became_peered":"2023-07-14T11:59:43.458631+0000","last_unstale":"2023-07-14T12:00:06.815209+0000","last_undegraded":"2023-07-14T12:00:06.815209+0000","last_fullsized":"2023-07-14T12:00:06.815209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:41:19.853895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805050+0000","last_change":"2023-07-14T11:59:47.776879+0000","last_active":"2023-07-14T12:00:06.805050+0000","last_peered":"2023-07-14T12:00:06.805050+0000","last_clean":"2023-07-14T12:00:06.805050+0000","last_became_active":"2023-07-14T11:59:47.776514+0000","last_became_peered":"2023-07-14T11:59:47.776514+0000","last_unstale":"2023-07-14T12:00:06.805050+0000","last_undegraded":"2023-07-14T12:00:06.805050+0000","last_fullsized":"2023-07-14T12:00:06.805050+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:14:32.460215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:13.464976+0000","last_change":"2023-07-14T11:59:42.754657+0000","last_active":"2023-07-14T12:00:13.464976+0000","last_peered":"2023-07-14T12:00:13.464976+0000","last_clean":"2023-07-14T12:00:13.464976+0000","last_became_active":"2023-07-14T11:59:42.754355+0000","last_became_peered":"2023-07-14T11:59:42.754355+0000","last_unstale":"2023-07-14T12:00:13.464976+0000","last_undegraded":"2023-07-14T12:00:13.464976+0000","last_fullsized":"2023-07-14T12:00:13.464976+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:16:59.044903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.329782+0000","last_change":"2023-07-14T12:00:07.329782+0000","last_active":"2023-07-14T12:00:07.329782+0000","last_peered":"2023-07-14T12:00:07.329782+0000","last_clean":"2023-07-14T12:00:07.329782+0000","last_became_active":"2023-07-14T12:00:07.329274+0000","last_became_peered":"2023-07-14T12:00:07.329274+0000","last_unstale":"2023-07-14T12:00:07.329782+0000","last_undegraded":"2023-07-14T12:00:07.329782+0000","last_fullsized":"2023-07-14T12:00:07.329782+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:49:01.837874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.807313+0000","last_change":"2023-07-14T11:59:47.916411+0000","last_active":"2023-07-14T12:00:06.807313+0000","last_peered":"2023-07-14T12:00:06.807313+0000","last_clean":"2023-07-14T12:00:06.807313+0000","last_became_active":"2023-07-14T11:59:47.915877+0000","last_became_peered":"2023-07-14T11:59:47.915877+0000","last_unstale":"2023-07-14T12:00:06.807313+0000","last_undegraded":"2023-07-14T12:00:06.807313+0000","last_fullsized":"2023-07-14T12:00:06.807313+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:08:05.471264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.804971+0000","last_change":"2023-07-14T11:59:42.769080+0000","last_active":"2023-07-14T12:00:06.804971+0000","last_peered":"2023-07-14T12:00:06.804971+0000","last_clean":"2023-07-14T12:00:06.804971+0000","last_became_active":"2023-07-14T11:59:42.768782+0000","last_became_peered":"2023-07-14T11:59:42.768782+0000","last_unstale":"2023-07-14T12:00:06.804971+0000","last_undegraded":"2023-07-14T12:00:06.804971+0000","last_fullsized":"2023-07-14T12:00:06.804971+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:31:48.128002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.806874+0000","last_change":"2023-07-14T11:59:43.460284+0000","last_active":"2023-07-14T12:00:06.806874+0000","last_peered":"2023-07-14T12:00:06.806874+0000","last_clean":"2023-07-14T12:00:06.806874+0000","last_became_active":"2023-07-14T11:59:43.459979+0000","last_became_peered":"2023-07-14T11:59:43.459979+0000","last_unstale":"2023-07-14T12:00:06.806874+0000","last_undegraded":"2023-07-14T12:00:06.806874+0000","last_fullsized":"2023-07-14T12:00:06.806874+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:25:15.513703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805201+0000","last_change":"2023-07-14T11:59:47.782359+0000","last_active":"2023-07-14T12:00:06.805201+0000","last_peered":"2023-07-14T12:00:06.805201+0000","last_clean":"2023-07-14T12:00:06.805201+0000","last_became_active":"2023-07-14T11:59:47.780484+0000","last_became_peered":"2023-07-14T11:59:47.780484+0000","last_unstale":"2023-07-14T12:00:06.805201+0000","last_undegraded":"2023-07-14T12:00:06.805201+0000","last_fullsized":"2023-07-14T12:00:06.805201+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:24:16.054488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.811311+0000","last_change":"2023-07-14T11:59:42.756823+0000","last_active":"2023-07-14T12:00:06.811311+0000","last_peered":"2023-07-14T12:00:06.811311+0000","last_clean":"2023-07-14T12:00:06.811311+0000","last_became_active":"2023-07-14T11:59:42.756412+0000","last_became_peered":"2023-07-14T11:59:42.756412+0000","last_unstale":"2023-07-14T12:00:06.811311+0000","last_undegraded":"2023-07-14T12:00:06.811311+0000","last_fullsized":"2023-07-14T12:00:06.811311+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:45:25.674809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.811156+0000","last_change":"2023-07-14T11:59:43.469151+0000","last_active":"2023-07-14T12:00:06.811156+0000","last_peered":"2023-07-14T12:00:06.811156+0000","last_clean":"2023-07-14T12:00:06.811156+0000","last_became_active":"2023-07-14T11:59:43.468835+0000","last_became_peered":"2023-07-14T11:59:43.468835+0000","last_unstale":"2023-07-14T12:00:06.811156+0000","last_undegraded":"2023-07-14T12:00:06.811156+0000","last_fullsized":"2023-07-14T12:00:06.811156+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:01:00.517226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.807655+0000","last_change":"2023-07-14T11:59:47.782663+0000","last_active":"2023-07-14T12:00:06.807655+0000","last_peered":"2023-07-14T12:00:06.807655+0000","last_clean":"2023-07-14T12:00:06.807655+0000","last_became_active":"2023-07-14T11:59:47.782175+0000","last_became_peered":"2023-07-14T11:59:47.782175+0000","last_unstale":"2023-07-14T12:00:06.807655+0000","last_undegraded":"2023-07-14T12:00:06.807655+0000","last_fullsized":"2023-07-14T12:00:06.807655+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:43:48.229057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805876+0000","last_change":"2023-07-14T11:59:42.759809+0000","last_active":"2023-07-14T12:00:06.805876+0000","last_peered":"2023-07-14T12:00:06.805876+0000","last_clean":"2023-07-14T12:00:06.805876+0000","last_became_active":"2023-07-14T11:59:42.759493+0000","last_became_peered":"2023-07-14T11:59:42.759493+0000","last_unstale":"2023-07-14T12:00:06.805876+0000","last_undegraded":"2023-07-14T12:00:06.805876+0000","last_fullsized":"2023-07-14T12:00:06.805876+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:21:22.391310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.810840+0000","last_change":"2023-07-14T11:59:43.465308+0000","last_active":"2023-07-14T12:00:06.810840+0000","last_peered":"2023-07-14T12:00:06.810840+0000","last_clean":"2023-07-14T12:00:06.810840+0000","last_became_active":"2023-07-14T11:59:43.464995+0000","last_became_peered":"2023-07-14T11:59:43.464995+0000","last_unstale":"2023-07-14T12:00:06.810840+0000","last_undegraded":"2023-07-14T12:00:06.810840+0000","last_fullsized":"2023-07-14T12:00:06.810840+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:22:41.954399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.807244+0000","last_change":"2023-07-14T11:59:47.916435+0000","last_active":"2023-07-14T12:00:06.807244+0000","last_peered":"2023-07-14T12:00:06.807244+0000","last_clean":"2023-07-14T12:00:06.807244+0000","last_became_active":"2023-07-14T11:59:47.915894+0000","last_became_peered":"2023-07-14T11:59:47.915894+0000","last_unstale":"2023-07-14T12:00:06.807244+0000","last_undegraded":"2023-07-14T12:00:06.807244+0000","last_fullsized":"2023-07-14T12:00:06.807244+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:15:10.755288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.807899+0000","last_change":"2023-07-14T11:59:42.760422+0000","last_active":"2023-07-14T12:00:06.807899+0000","last_peered":"2023-07-14T12:00:06.807899+0000","last_clean":"2023-07-14T12:00:06.807899+0000","last_became_active":"2023-07-14T11:59:42.758548+0000","last_became_peered":"2023-07-14T11:59:42.758548+0000","last_unstale":"2023-07-14T12:00:06.807899+0000","last_undegraded":"2023-07-14T12:00:06.807899+0000","last_fullsized":"2023-07-14T12:00:06.807899+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:00:42.799391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.328110+0000","last_change":"2023-07-14T11:59:43.455296+0000","last_active":"2023-07-14T12:00:07.328110+0000","last_peered":"2023-07-14T12:00:07.328110+0000","last_clean":"2023-07-14T12:00:07.328110+0000","last_became_active":"2023-07-14T11:59:43.455003+0000","last_became_peered":"2023-07-14T11:59:43.455003+0000","last_unstale":"2023-07-14T12:00:07.328110+0000","last_undegraded":"2023-07-14T12:00:07.328110+0000","last_fullsized":"2023-07-14T12:00:07.328110+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:43:29.555694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805023+0000","last_change":"2023-07-14T11:59:47.779262+0000","last_active":"2023-07-14T12:00:06.805023+0000","last_peered":"2023-07-14T12:00:06.805023+0000","last_clean":"2023-07-14T12:00:06.805023+0000","last_became_active":"2023-07-14T11:59:47.778863+0000","last_became_peered":"2023-07-14T11:59:47.778863+0000","last_unstale":"2023-07-14T12:00:06.805023+0000","last_undegraded":"2023-07-14T12:00:06.805023+0000","last_fullsized":"2023-07-14T12:00:06.805023+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:11:16.752296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.806219+0000","last_change":"2023-07-14T11:59:42.768252+0000","last_active":"2023-07-14T12:00:06.806219+0000","last_peered":"2023-07-14T12:00:06.806219+0000","last_clean":"2023-07-14T12:00:06.806219+0000","last_became_active":"2023-07-14T11:59:42.767956+0000","last_became_peered":"2023-07-14T11:59:42.767956+0000","last_unstale":"2023-07-14T12:00:06.806219+0000","last_undegraded":"2023-07-14T12:00:06.806219+0000","last_fullsized":"2023-07-14T12:00:06.806219+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:19:59.247002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.328734+0000","last_change":"2023-07-14T11:59:43.461899+0000","last_active":"2023-07-14T12:00:07.328734+0000","last_peered":"2023-07-14T12:00:07.328734+0000","last_clean":"2023-07-14T12:00:07.328734+0000","last_became_active":"2023-07-14T11:59:43.461535+0000","last_became_peered":"2023-07-14T11:59:43.461535+0000","last_unstale":"2023-07-14T12:00:07.328734+0000","last_undegraded":"2023-07-14T12:00:07.328734+0000","last_fullsized":"2023-07-14T12:00:07.328734+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:43:24.603371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.812720+0000","last_change":"2023-07-14T11:59:47.782964+0000","last_active":"2023-07-14T12:00:06.812720+0000","last_peered":"2023-07-14T12:00:06.812720+0000","last_clean":"2023-07-14T12:00:06.812720+0000","last_became_active":"2023-07-14T11:59:47.782563+0000","last_became_peered":"2023-07-14T11:59:47.782563+0000","last_unstale":"2023-07-14T12:00:06.812720+0000","last_undegraded":"2023-07-14T12:00:06.812720+0000","last_fullsized":"2023-07-14T12:00:06.812720+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:27:49.195128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808938+0000","last_change":"2023-07-14T11:59:42.751355+0000","last_active":"2023-07-14T12:00:06.808938+0000","last_peered":"2023-07-14T12:00:06.808938+0000","last_clean":"2023-07-14T12:00:06.808938+0000","last_became_active":"2023-07-14T11:59:42.751063+0000","last_became_peered":"2023-07-14T11:59:42.751063+0000","last_unstale":"2023-07-14T12:00:06.808938+0000","last_undegraded":"2023-07-14T12:00:06.808938+0000","last_fullsized":"2023-07-14T12:00:06.808938+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:28:15.966689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.328252+0000","last_change":"2023-07-14T11:59:43.465891+0000","last_active":"2023-07-14T12:00:07.328252+0000","last_peered":"2023-07-14T12:00:07.328252+0000","last_clean":"2023-07-14T12:00:07.328252+0000","last_became_active":"2023-07-14T11:59:43.465575+0000","last_became_peered":"2023-07-14T11:59:43.465575+0000","last_unstale":"2023-07-14T12:00:07.328252+0000","last_undegraded":"2023-07-14T12:00:07.328252+0000","last_fullsized":"2023-07-14T12:00:07.328252+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:43:52.704477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.809069+0000","last_change":"2023-07-14T11:59:47.777547+0000","last_active":"2023-07-14T12:00:06.809069+0000","last_peered":"2023-07-14T12:00:06.809069+0000","last_clean":"2023-07-14T12:00:06.809069+0000","last_became_active":"2023-07-14T11:59:47.777150+0000","last_became_peered":"2023-07-14T11:59:47.777150+0000","last_unstale":"2023-07-14T12:00:06.809069+0000","last_undegraded":"2023-07-14T12:00:06.809069+0000","last_fullsized":"2023-07-14T12:00:06.809069+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:21:35.638026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.809105+0000","last_change":"2023-07-14T11:59:42.761236+0000","last_active":"2023-07-14T12:00:06.809105+0000","last_peered":"2023-07-14T12:00:06.809105+0000","last_clean":"2023-07-14T12:00:06.809105+0000","last_became_active":"2023-07-14T11:59:42.760893+0000","last_became_peered":"2023-07-14T11:59:42.760893+0000","last_unstale":"2023-07-14T12:00:06.809105+0000","last_undegraded":"2023-07-14T12:00:06.809105+0000","last_fullsized":"2023-07-14T12:00:06.809105+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:11:17.477118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.328670+0000","last_change":"2023-07-14T11:59:43.462518+0000","last_active":"2023-07-14T12:00:07.328670+0000","last_peered":"2023-07-14T12:00:07.328670+0000","last_clean":"2023-07-14T12:00:07.328670+0000","last_became_active":"2023-07-14T11:59:43.462178+0000","last_became_peered":"2023-07-14T11:59:43.462178+0000","last_unstale":"2023-07-14T12:00:07.328670+0000","last_undegraded":"2023-07-14T12:00:07.328670+0000","last_fullsized":"2023-07-14T12:00:07.328670+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:09:38.398771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.811302+0000","last_change":"2023-07-14T11:59:43.474142+0000","last_active":"2023-07-14T12:00:06.811302+0000","last_peered":"2023-07-14T12:00:06.811302+0000","last_clean":"2023-07-14T12:00:06.811302+0000","last_became_active":"2023-07-14T11:59:43.473863+0000","last_became_peered":"2023-07-14T11:59:43.473863+0000","last_unstale":"2023-07-14T12:00:06.811302+0000","last_undegraded":"2023-07-14T12:00:06.811302+0000","last_fullsized":"2023-07-14T12:00:06.811302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:34:40.849796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805367+0000","last_change":"2023-07-14T11:59:43.462193+0000","last_active":"2023-07-14T12:00:06.805367+0000","last_peered":"2023-07-14T12:00:06.805367+0000","last_clean":"2023-07-14T12:00:06.805367+0000","last_became_active":"2023-07-14T11:59:43.461390+0000","last_became_peered":"2023-07-14T11:59:43.461390+0000","last_unstale":"2023-07-14T12:00:06.805367+0000","last_undegraded":"2023-07-14T12:00:06.805367+0000","last_fullsized":"2023-07-14T12:00:06.805367+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:43:44.553075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805098+0000","last_change":"2023-07-14T11:59:43.467561+0000","last_active":"2023-07-14T12:00:06.805098+0000","last_peered":"2023-07-14T12:00:06.805098+0000","last_clean":"2023-07-14T12:00:06.805098+0000","last_became_active":"2023-07-14T11:59:43.467256+0000","last_became_peered":"2023-07-14T11:59:43.467256+0000","last_unstale":"2023-07-14T12:00:06.805098+0000","last_undegraded":"2023-07-14T12:00:06.805098+0000","last_fullsized":"2023-07-14T12:00:06.805098+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:04:06.326191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.813046+0000","last_change":"2023-07-14T11:59:43.472405+0000","last_active":"2023-07-14T12:00:06.813046+0000","last_peered":"2023-07-14T12:00:06.813046+0000","last_clean":"2023-07-14T12:00:06.813046+0000","last_became_active":"2023-07-14T11:59:43.471863+0000","last_became_peered":"2023-07-14T11:59:43.471863+0000","last_unstale":"2023-07-14T12:00:06.813046+0000","last_undegraded":"2023-07-14T12:00:06.813046+0000","last_fullsized":"2023-07-14T12:00:06.813046+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:22:54.987390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808865+0000","last_change":"2023-07-14T11:59:43.469966+0000","last_active":"2023-07-14T12:00:06.808865+0000","last_peered":"2023-07-14T12:00:06.808865+0000","last_clean":"2023-07-14T12:00:06.808865+0000","last_became_active":"2023-07-14T11:59:43.469540+0000","last_became_peered":"2023-07-14T11:59:43.469540+0000","last_unstale":"2023-07-14T12:00:06.808865+0000","last_undegraded":"2023-07-14T12:00:06.808865+0000","last_fullsized":"2023-07-14T12:00:06.808865+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:19:04.529881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.812535+0000","last_change":"2023-07-14T11:59:43.465032+0000","last_active":"2023-07-14T12:00:06.812535+0000","last_peered":"2023-07-14T12:00:06.812535+0000","last_clean":"2023-07-14T12:00:06.812535+0000","last_became_active":"2023-07-14T11:59:43.464689+0000","last_became_peered":"2023-07-14T11:59:43.464689+0000","last_unstale":"2023-07-14T12:00:06.812535+0000","last_undegraded":"2023-07-14T12:00:06.812535+0000","last_fullsized":"2023-07-14T12:00:06.812535+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:15:12.845214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.806463+0000","last_change":"2023-07-14T11:59:43.463042+0000","last_active":"2023-07-14T12:00:06.806463+0000","last_peered":"2023-07-14T12:00:06.806463+0000","last_clean":"2023-07-14T12:00:06.806463+0000","last_became_active":"2023-07-14T11:59:43.462665+0000","last_became_peered":"2023-07-14T11:59:43.462665+0000","last_unstale":"2023-07-14T12:00:06.806463+0000","last_undegraded":"2023-07-14T12:00:06.806463+0000","last_fullsized":"2023-07-14T12:00:06.806463+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:50:19.141988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.816236+0000","last_change":"2023-07-14T12:00:06.816236+0000","last_active":"2023-07-14T12:00:06.816236+0000","last_peered":"2023-07-14T12:00:06.816236+0000","last_clean":"2023-07-14T12:00:06.816236+0000","last_became_active":"2023-07-14T12:00:06.815733+0000","last_became_peered":"2023-07-14T12:00:06.815733+0000","last_unstale":"2023-07-14T12:00:06.816236+0000","last_undegraded":"2023-07-14T12:00:06.816236+0000","last_fullsized":"2023-07-14T12:00:06.816236+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:14:57.114402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.809719+0000","last_change":"2023-07-14T11:59:43.470256+0000","last_active":"2023-07-14T12:00:06.809719+0000","last_peered":"2023-07-14T12:00:06.809719+0000","last_clean":"2023-07-14T12:00:06.809719+0000","last_became_active":"2023-07-14T11:59:43.469861+0000","last_became_peered":"2023-07-14T11:59:43.469861+0000","last_unstale":"2023-07-14T12:00:06.809719+0000","last_undegraded":"2023-07-14T12:00:06.809719+0000","last_fullsized":"2023-07-14T12:00:06.809719+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:54:04.221673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.811456+0000","last_change":"2023-07-14T11:59:43.458009+0000","last_active":"2023-07-14T12:00:06.811456+0000","last_peered":"2023-07-14T12:00:06.811456+0000","last_clean":"2023-07-14T12:00:06.811456+0000","last_became_active":"2023-07-14T11:59:43.457657+0000","last_became_peered":"2023-07-14T11:59:43.457657+0000","last_unstale":"2023-07-14T12:00:06.811456+0000","last_undegraded":"2023-07-14T12:00:06.811456+0000","last_fullsized":"2023-07-14T12:00:06.811456+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:14:41.854547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805358+0000","last_change":"2023-07-14T11:59:43.457568+0000","last_active":"2023-07-14T12:00:06.805358+0000","last_peered":"2023-07-14T12:00:06.805358+0000","last_clean":"2023-07-14T12:00:06.805358+0000","last_became_active":"2023-07-14T11:59:43.457203+0000","last_became_peered":"2023-07-14T11:59:43.457203+0000","last_unstale":"2023-07-14T12:00:06.805358+0000","last_undegraded":"2023-07-14T12:00:06.805358+0000","last_fullsized":"2023-07-14T12:00:06.805358+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:14:37.939509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808045+0000","last_change":"2023-07-14T11:59:43.470840+0000","last_active":"2023-07-14T12:00:06.808045+0000","last_peered":"2023-07-14T12:00:06.808045+0000","last_clean":"2023-07-14T12:00:06.808045+0000","last_became_active":"2023-07-14T11:59:43.470531+0000","last_became_peered":"2023-07-14T11:59:43.470531+0000","last_unstale":"2023-07-14T12:00:06.808045+0000","last_undegraded":"2023-07-14T12:00:06.808045+0000","last_fullsized":"2023-07-14T12:00:06.808045+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:49:50.862808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.817057+0000","last_change":"2023-07-14T12:00:06.817057+0000","last_active":"2023-07-14T12:00:06.817057+0000","last_peered":"2023-07-14T12:00:06.817057+0000","last_clean":"2023-07-14T12:00:06.817057+0000","last_became_active":"2023-07-14T12:00:06.816680+0000","last_became_peered":"2023-07-14T12:00:06.816680+0000","last_unstale":"2023-07-14T12:00:06.817057+0000","last_undegraded":"2023-07-14T12:00:06.817057+0000","last_fullsized":"2023-07-14T12:00:06.817057+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:22:01.363770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284390,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6412,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":16932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416401,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":970752,"data_stored":486215,"data_compressed":17563,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:59:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.50700000000000001}]},{"osd":1,"last update":"Fri Jul 14 11:59:56 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.71399999999999997}]},{"osd":2,"last update":"Fri Jul 14 11:59:56 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.377}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":4,"last update":"Fri Jul 14 11:59:56 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.61899999999999999}]},{"osd":5,"last update":"Fri Jul 14 11:59:56 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.54200000000000004}]},{"osd":6,"last update":"Fri Jul 14 11:59:56 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":2.1869999999999998,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":0.65200000000000002}]},{"osd":7,"last update":"Fri Jul 14 11:59:56 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.79000000000000004}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":9,"last update":"Fri Jul 14 11:59:56 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":2.0339999999999998,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.83099999999999996}]},{"osd":10,"last update":"Fri Jul 14 11:59:56 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":2.1019999999999999,"5min":2.1019999999999999,"15min":2.1019999999999999},"last":0.68200000000000005}]}]},{"osd":10,"up_from":72,"seq":309237645333,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":1016,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":1040384,"data_stored":1080240,"data_compressed":53729,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":1,"last update":"Fri Jul 14 11:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.435},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.48499999999999999}]},{"osd":2,"last update":"Fri Jul 14 11:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.628},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.67700000000000005}]},{"osd":3,"last update":"Fri Jul 14 11:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.58399999999999996}]},{"osd":4,"last update":"Fri Jul 14 11:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.59899999999999998}]},{"osd":5,"last update":"Fri Jul 14 11:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.61299999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.86199999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.60899999999999999}]},{"osd":8,"last update":"Fri Jul 14 11:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.75900000000000001}]},{"osd":9,"last update":"Fri Jul 14 11:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.69999999999999996}]},{"osd":11,"last update":"Fri Jul 14 11:59:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.79900000000000004}]}]},{"osd":9,"up_from":67,"seq":287762808858,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":974848,"data_stored":486797,"data_compressed":17563,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.85999999999999999}]},{"osd":1,"last update":"Fri Jul 14 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.96599999999999997}]},{"osd":2,"last update":"Fri Jul 14 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.91100000000000003}]},{"osd":3,"last update":"Fri Jul 14 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.88700000000000001}]},{"osd":4,"last update":"Fri Jul 14 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":2.0099999999999998,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":0.93500000000000005}]},{"osd":5,"last update":"Fri Jul 14 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.754},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.77800000000000002}]},{"osd":6,"last update":"Fri Jul 14 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":1.006}]},{"osd":7,"last update":"Fri Jul 14 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.95399999999999996}]},{"osd":8,"last update":"Fri Jul 14 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":1.105},{"interface":"front","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":1.079}]},{"osd":10,"last update":"Fri Jul 14 11:59:35 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":1.2130000000000001},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":1.1950000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1759999999999999}]}]},{"osd":8,"up_from":59,"seq":253403070495,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":978944,"data_stored":486819,"data_compressed":17563,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:59:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.60599999999999998,"15min":0.60999999999999999},"min":{"1min":0.42299999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.877,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.57899999999999996,"15min":0.58499999999999996},"min":{"1min":0.376,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81299999999999994,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.60599999999999998}]},{"osd":1,"last update":"Fri Jul 14 11:59:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58699999999999997,"15min":0.58099999999999996},"min":{"1min":0.41599999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.73099999999999998,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.63,"5min":0.627,"15min":0.627},"min":{"1min":0.437,"5min":0.376,"15min":0.376},"max":{"1min":0.80900000000000005,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.75}]},{"osd":2,"last update":"Fri Jul 14 11:59:49 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.34499999999999997,"5min":0.309,"15min":0.309},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.51100000000000001,"15min":0.51800000000000002},"min":{"1min":0.317,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.60699999999999998,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.54600000000000004}]},{"osd":3,"last update":"Fri Jul 14 11:59:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.63500000000000001,"15min":0.64300000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.878,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.65000000000000002,"15min":0.65400000000000003},"min":{"1min":0.44500000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.627}]},{"osd":4,"last update":"Fri Jul 14 11:59:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.61599999999999999,"15min":0.62},"min":{"1min":0.48399999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.77700000000000002,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.64900000000000002,"15min":0.64500000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84599999999999997,"5min":0.87,"15min":0.87},"last":0.76300000000000001}]},{"osd":5,"last update":"Fri Jul 14 11:59:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.61499999999999999,"15min":0.62},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.79600000000000004,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.37,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72599999999999998,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.51800000000000002}]},{"osd":6,"last update":"Fri Jul 14 11:59:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.74199999999999999,"15min":0.749},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.874,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.748,"5min":0.78700000000000003,"15min":0.79300000000000004},"min":{"1min":0.52300000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.92600000000000005,"5min":1.087,"15min":1.087},"last":0.81699999999999995}]},{"osd":7,"last update":"Fri Jul 14 11:59:49 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.751,"15min":0.753},"min":{"1min":0.502,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.99199999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.73799999999999999,"15min":0.746},"min":{"1min":0.51400000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.89300000000000002,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.79700000000000004}]},{"osd":9,"last update":"Fri Jul 14 11:59:17 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.878}]},{"osd":10,"last update":"Fri Jul 14 11:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.86499999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]}]},{"osd":7,"up_from":52,"seq":223338299428,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":1004,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":1028096,"data_stored":1077408,"data_compressed":53729,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.443,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.88300000000000001,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59699999999999998,"15min":0.59599999999999997},"min":{"1min":0.50700000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.495}]},{"osd":1,"last update":"Fri Jul 14 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.42099999999999999,"15min":0.41499999999999998},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.70199999999999996,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.439,"5min":0.44700000000000001,"15min":0.44800000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.41599999999999998}]},{"osd":2,"last update":"Fri Jul 14 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59499999999999997,"15min":0.59699999999999998},"min":{"1min":0.433,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77500000000000002,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.625,"15min":0.61699999999999999},"min":{"1min":0.48999999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.70299999999999996}]},{"osd":3,"last update":"Fri Jul 14 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.63200000000000001,"15min":0.629},"min":{"1min":0.47099999999999997,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.66000000000000003,"15min":0.65800000000000003},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.56299999999999994}]},{"osd":4,"last update":"Fri Jul 14 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.51000000000000001,"15min":0.501},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.52700000000000002,"15min":0.53100000000000003},"min":{"1min":0.35999999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.72799999999999998,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.5}]},{"osd":5,"last update":"Fri Jul 14 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.71799999999999997,"15min":0.72199999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.98899999999999999,"5min":1.341,"15min":1.341},"last":0.79900000000000004}]},{"osd":6,"last update":"Fri Jul 14 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.72199999999999998,"15min":0.71999999999999997},"min":{"1min":0.53100000000000003,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.69499999999999995,"15min":0.69699999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.85799999999999998,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.65700000000000003}]},{"osd":8,"last update":"Fri Jul 14 11:59:50 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.71199999999999997,"15min":0.69999999999999996},"min":{"1min":0.59999999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.70699999999999996,"15min":0.70099999999999996},"min":{"1min":0.57699999999999996,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.88100000000000001,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.76300000000000001}]},{"osd":9,"last update":"Fri Jul 14 11:59:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.72699999999999998}]},{"osd":10,"last update":"Fri Jul 14 11:59:35 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.77500000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]}]},{"osd":6,"up_from":47,"seq":201863462951,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":970752,"data_stored":486215,"data_compressed":17563,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 12:00:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.48699999999999999,"15min":0.47299999999999998},"min":{"1min":0.39100000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.60599999999999998,"15min":0.59799999999999998},"min":{"1min":0.46600000000000003,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.82299999999999995}]},{"osd":1,"last update":"Fri Jul 14 12:00:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.65600000000000003,"15min":0.64600000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":1.056},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.72699999999999998,"15min":0.72199999999999998},"min":{"1min":0.51200000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":1.109}]},{"osd":2,"last update":"Fri Jul 14 12:00:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.65500000000000003,"15min":0.64500000000000002},"min":{"1min":0.48999999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.93799999999999994,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.70099999999999996,"15min":0.68999999999999995},"min":{"1min":0.41399999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0049999999999999,"5min":1.155,"15min":1.155},"last":0.92500000000000004}]},{"osd":3,"last update":"Fri Jul 14 12:00:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.61399999999999999,"15min":0.59999999999999998},"min":{"1min":0.46200000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.93600000000000005,"5min":1.107,"15min":1.107},"last":0.75},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.70899999999999996,"15min":0.69699999999999995},"min":{"1min":0.57499999999999996,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91500000000000004,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":1.0329999999999999}]},{"osd":4,"last update":"Fri Jul 14 12:00:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.68500000000000005,"15min":0.67300000000000004},"min":{"1min":0.441,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.754,"15min":0.73999999999999999},"min":{"1min":0.57199999999999995,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":1.1519999999999999}]},{"osd":5,"last update":"Fri Jul 14 12:00:06 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.79900000000000004,"15min":0.79300000000000004},"min":{"1min":0.67600000000000005,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1000000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.745,"15min":0.73399999999999999},"min":{"1min":0.55000000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":1.0049999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.77100000000000002,"15min":0.752},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":1.169},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.73599999999999999,"15min":0.71499999999999997},"min":{"1min":0.67000000000000004,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0629999999999999,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":1.179}]},{"osd":8,"last update":"Fri Jul 14 11:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.81000000000000005,"15min":0.80300000000000005},"min":{"1min":0.63400000000000001,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":1.2030000000000001},{"interface":"front","average":{"1min":0.876,"5min":0.82099999999999995,"15min":0.81200000000000006},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.052,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":1.127}]},{"osd":9,"last update":"Fri Jul 14 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":1.23},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":1.2569999999999999}]},{"osd":10,"last update":"Fri Jul 14 11:59:35 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":1.218},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":1.278}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3300000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":983040,"data_stored":490009,"data_compressed":17563,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:59:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.55100000000000005,"15min":0.52900000000000003},"min":{"1min":0.41499999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.65400000000000003,"15min":0.73999999999999999},"min":{"1min":0.39300000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.90200000000000002,"5min":5.0940000000000003,"15min":5.0940000000000003},"last":0.56299999999999994}]},{"osd":2,"last update":"Fri Jul 14 11:59:37 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.53000000000000003,"15min":0.498},"min":{"1min":0.41899999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.64300000000000002,"15min":0.70299999999999996},"min":{"1min":0.47299999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.82499999999999996,"5min":5.0570000000000004,"15min":5.0570000000000004},"last":0.72999999999999998}]},{"osd":3,"last update":"Fri Jul 14 12:00:00 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60699999999999998,"15min":0.59699999999999998},"min":{"1min":0.39900000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.95999999999999996,"5min":1.054,"15min":1.054},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.42399999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.92600000000000005,"5min":1.006,"15min":1.006},"last":0.67500000000000004}]},{"osd":4,"last update":"Fri Jul 14 11:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.437,"15min":0.42099999999999999},"min":{"1min":0.36699999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1,"5min":1,"15min":1},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.46000000000000002,"15min":0.45100000000000001},"min":{"1min":0.34999999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.61199999999999999}]},{"osd":5,"last update":"Fri Jul 14 11:59:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.60899999999999999,"15min":0.58499999999999996},"min":{"1min":0.50900000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.83399999999999996,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.58399999999999996,"15min":0.55700000000000005},"min":{"1min":0.53600000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.81100000000000005,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.75700000000000001}]},{"osd":6,"last update":"Fri Jul 14 12:00:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.65700000000000003,"15min":0.63900000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.69399999999999995,"15min":0.67500000000000004},"min":{"1min":0.53800000000000003,"5min":0.44,"15min":0.44},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.77200000000000002}]},{"osd":7,"last update":"Fri Jul 14 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.54300000000000004,"15min":0.53200000000000003},"min":{"1min":0.39700000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.61799999999999999,"15min":0.59399999999999997},"min":{"1min":0.59899999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.85599999999999998}]},{"osd":8,"last update":"Fri Jul 14 11:59:49 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.71199999999999997,"15min":0.70099999999999996},"min":{"1min":0.624,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68999999999999995,"15min":0.68600000000000005},"min":{"1min":0.52700000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.84399999999999997}]},{"osd":9,"last update":"Fri Jul 14 11:59:16 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.746}]},{"osd":10,"last update":"Fri Jul 14 11:59:37 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":0.82699999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738435,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":974848,"data_stored":486305,"data_compressed":17563,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 11:59:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56399999999999995,"15min":0.54200000000000004},"min":{"1min":0.33100000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.96399999999999997,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.55000000000000004,"15min":0.52800000000000002},"min":{"1min":0.436,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.83299999999999996}]},{"osd":2,"last update":"Fri Jul 14 11:59:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.61699999999999999,"15min":0.58699999999999997},"min":{"1min":0.43099999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.0569999999999999,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.59299999999999997,"15min":0.56100000000000005},"min":{"1min":0.42199999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.84699999999999998,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.60299999999999998}]},{"osd":3,"last update":"Fri Jul 14 11:58:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.54500000000000004,"15min":0.52300000000000002},"min":{"1min":0.313,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.55700000000000005,"15min":0.51100000000000001},"min":{"1min":0.39600000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.375}]},{"osd":4,"last update":"Fri Jul 14 11:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.60799999999999998,"15min":0.58199999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.62,"15min":0.58499999999999996},"min":{"1min":0.40999999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.93200000000000005}]},{"osd":5,"last update":"Fri Jul 14 11:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.65900000000000003,"15min":0.64600000000000002},"min":{"1min":0.40500000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.96899999999999997,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.69599999999999995,"15min":0.67600000000000005},"min":{"1min":0.5,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.99099999999999999,"5min":1.131,"15min":1.131},"last":0.77800000000000002}]},{"osd":6,"last update":"Fri Jul 14 12:00:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.67600000000000005,"15min":0.66000000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.92600000000000005,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.68700000000000006,"15min":0.68100000000000005},"min":{"1min":0.40999999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86899999999999999,"5min":1.081,"15min":1.081},"last":0.70799999999999996}]},{"osd":7,"last update":"Fri Jul 14 11:59:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.72399999999999998,"15min":0.70999999999999996},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.88},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.75900000000000001,"15min":0.746},"min":{"1min":0.47199999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.90100000000000002}]},{"osd":8,"last update":"Fri Jul 14 11:59:48 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.78800000000000003,"15min":0.79600000000000004},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.93200000000000005,"5min":1.05,"15min":1.05},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81499999999999995,"15min":0.81599999999999995},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.80100000000000005}]},{"osd":9,"last update":"Fri Jul 14 11:59:13 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.97299999999999998}]},{"osd":10,"last update":"Fri Jul 14 11:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":1.008},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":1.0389999999999999}]},{"osd":11,"last update":"Fri Jul 14 11:59:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":1.115},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":1.1040000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":987136,"data_stored":490043,"data_compressed":17563,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.55300000000000005,"15min":0.501},"min":{"1min":0.39500000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.745,"5min":6.54,"15min":6.54},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.52100000000000002,"15min":0.497},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78100000000000003,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.56699999999999995}]},{"osd":1,"last update":"Fri Jul 14 11:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.52300000000000002,"15min":0.49199999999999999},"min":{"1min":0.39600000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.746},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.53500000000000003,"15min":0.5},"min":{"1min":0.373,"5min":0.31,"15min":0.31},"max":{"1min":0.91200000000000003,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.67800000000000005}]},{"osd":3,"last update":"Fri Jul 14 12:00:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.58499999999999996,"15min":0.54600000000000004},"min":{"1min":0.39200000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.83299999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.502},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.59599999999999997,"15min":0.55900000000000005},"min":{"1min":0.41599999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.88400000000000001,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.65400000000000003}]},{"osd":4,"last update":"Fri Jul 14 11:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.55000000000000004,"15min":0.53200000000000003},"min":{"1min":0.43099999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.55400000000000005,"15min":0.54300000000000004},"min":{"1min":0.39600000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.82999999999999996,"5min":0.873,"15min":0.873},"last":0.70899999999999996}]},{"osd":5,"last update":"Fri Jul 14 11:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.44400000000000001,"15min":0.435},"min":{"1min":0.313,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.46500000000000002,"15min":0.45000000000000001},"min":{"1min":0.28199999999999997,"5min":0.252,"15min":0.252},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.47099999999999997}]},{"osd":6,"last update":"Fri Jul 14 12:00:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.65200000000000002,"15min":0.63700000000000001},"min":{"1min":0.50700000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.64800000000000002,"15min":0.63800000000000001},"min":{"1min":0.46600000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.72899999999999998}]},{"osd":7,"last update":"Fri Jul 14 11:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.60999999999999999,"15min":0.60599999999999998},"min":{"1min":0.42799999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.79000000000000004,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.66800000000000004,"15min":0.67200000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.89900000000000002,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.79500000000000004}]},{"osd":8,"last update":"Fri Jul 14 11:59:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.59199999999999997,"15min":0.59799999999999998},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.93500000000000005,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.621,"15min":0.621},"min":{"1min":0.371,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.63300000000000001}]},{"osd":9,"last update":"Fri Jul 14 11:59:17 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.86599999999999999}]},{"osd":10,"last update":"Fri Jul 14 11:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.78100000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]}]},{"osd":3,"up_from":26,"seq":111669149750,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":978944,"data_stored":486319,"data_compressed":17563,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:59:56 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.42699999999999999,"15min":0.377},"min":{"1min":0.32900000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.45200000000000001,"15min":0.40100000000000002},"min":{"1min":0.32800000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.64200000000000002,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.40699999999999997}]},{"osd":1,"last update":"Fri Jul 14 11:59:56 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.64200000000000002,"15min":0.57299999999999995},"min":{"1min":0.44600000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0249999999999999,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.58499999999999996,"15min":0.54700000000000004},"min":{"1min":0.40000000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.94699999999999995,"5min":1.111,"15min":1.111},"last":0.78600000000000003}]},{"osd":2,"last update":"Fri Jul 14 11:59:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.65400000000000003,"15min":0.59999999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.89300000000000002,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.753},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.65500000000000003,"15min":0.60699999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.94399999999999995,"5min":1.206,"15min":1.206},"last":0.76000000000000001}]},{"osd":4,"last update":"Fri Jul 14 11:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.63600000000000001,"15min":0.60399999999999998},"min":{"1min":0.437,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.65300000000000002,"15min":0.61199999999999999},"min":{"1min":0.48299999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0860000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.89900000000000002}]},{"osd":5,"last update":"Fri Jul 14 11:59:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64100000000000001,"15min":0.623},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.89400000000000002,"5min":1.05,"15min":1.05},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.71799999999999997,"15min":0.68400000000000005},"min":{"1min":0.52300000000000002,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0189999999999999,"5min":1.077,"15min":1.077},"last":0.71099999999999997}]},{"osd":6,"last update":"Fri Jul 14 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.749,"15min":0.71799999999999997},"min":{"1min":0.53300000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.97999999999999998,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.69499999999999995,"15min":0.69399999999999995},"min":{"1min":0.44700000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.95699999999999996,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.56399999999999995}]},{"osd":7,"last update":"Fri Jul 14 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.86199999999999999,"15min":0.86599999999999999},"min":{"1min":0.62,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.113,"5min":1.254,"15min":1.254},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.89300000000000002,"15min":0.88800000000000001},"min":{"1min":0.64000000000000001,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.173,"5min":1.383,"15min":1.383},"last":0.94099999999999995}]},{"osd":8,"last update":"Fri Jul 14 11:58:46 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.82799999999999996}]},{"osd":9,"last update":"Fri Jul 14 11:59:16 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.91700000000000004}]},{"osd":10,"last update":"Fri Jul 14 11:59:35 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.871},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.84099999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]}]},{"osd":4,"up_from":34,"seq":146028888114,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":974848,"data_stored":486237,"data_compressed":17563,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.52700000000000002,"15min":0.504},"min":{"1min":0.41399999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.53700000000000003,"15min":0.52300000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.80000000000000004,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.48299999999999998}]},{"osd":1,"last update":"Fri Jul 14 11:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.36199999999999999,"15min":0.35099999999999998},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.377,"15min":0.372},"min":{"1min":0.23999999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.57799999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.38800000000000001}]},{"osd":2,"last update":"Fri Jul 14 11:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54300000000000004,"15min":0.53700000000000003},"min":{"1min":0.39100000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.57199999999999995,"15min":0.55000000000000004},"min":{"1min":0.38800000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.58999999999999997}]},{"osd":3,"last update":"Fri Jul 14 11:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.56699999999999995,"15min":0.54900000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85399999999999998,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.57599999999999996,"15min":0.56399999999999995},"min":{"1min":0.47099999999999997,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.75800000000000001,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.80500000000000005}]},{"osd":5,"last update":"Fri Jul 14 11:59:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.58999999999999997,"15min":0.57799999999999996},"min":{"1min":0.41199999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.85399999999999998,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58999999999999997,"15min":0.58599999999999997},"min":{"1min":0.42599999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81699999999999995,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.51000000000000001}]},{"osd":6,"last update":"Fri Jul 14 12:00:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.623,"15min":0.61599999999999999},"min":{"1min":0.46800000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.80800000000000005,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.69199999999999995,"15min":0.68400000000000005},"min":{"1min":0.48299999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.86099999999999999}]},{"osd":7,"last update":"Fri Jul 14 11:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.55400000000000005,"15min":0.54300000000000004},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.53100000000000003,"15min":0.51600000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.47299999999999998}]},{"osd":8,"last update":"Fri Jul 14 11:59:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.61699999999999999,"15min":0.61499999999999999},"min":{"1min":0.46600000000000003,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.84599999999999997,"5min":0.874,"15min":0.874},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.51800000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.032,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.78800000000000003}]},{"osd":9,"last update":"Fri Jul 14 11:59:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.92800000000000005}]},{"osd":10,"last update":"Fri Jul 14 11:59:37 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.64000000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]}]},{"osd":5,"up_from":39,"seq":167503724590,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":1044480,"data_stored":1077680,"data_compressed":53729,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:59:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60399999999999998,"15min":0.60499999999999998},"min":{"1min":0.36699999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.78600000000000003,"5min":1.018,"15min":1.018},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.59899999999999998,"15min":0.60899999999999999},"min":{"1min":0.34599999999999997,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.81299999999999994,"5min":1.127,"15min":1.127},"last":0.54000000000000004}]},{"osd":1,"last update":"Fri Jul 14 11:59:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.54800000000000004,"15min":0.52600000000000002},"min":{"1min":0.41199999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.875,"5min":1.018,"15min":1.018},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.54800000000000004,"15min":0.53200000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.82299999999999995,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.65200000000000002}]},{"osd":2,"last update":"Fri Jul 14 11:59:37 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.38600000000000001,"15min":0.378},"min":{"1min":0.26800000000000002,"5min":0.22,"15min":0.22},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.41099999999999998,"15min":0.40799999999999997},"min":{"1min":0.309,"5min":0.253,"15min":0.253},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.58599999999999997}]},{"osd":3,"last update":"Fri Jul 14 11:59:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.65400000000000003,"15min":0.64400000000000002},"min":{"1min":0.40000000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.77700000000000002,"5min":3.6640000000000001,"15min":3.6640000000000001},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.65500000000000003,"15min":0.65600000000000003},"min":{"1min":0.42699999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.82399999999999995,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.90900000000000003}]},{"osd":4,"last update":"Fri Jul 14 11:59:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56599999999999995,"15min":0.55200000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.75900000000000001,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57399999999999995,"15min":0.56499999999999995},"min":{"1min":0.39900000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.84199999999999997,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":1.0009999999999999}]},{"osd":6,"last update":"Fri Jul 14 12:00:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.67400000000000004,"15min":0.67000000000000004},"min":{"1min":0.46800000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.91800000000000004,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":1.351},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.69699999999999995,"15min":0.69599999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.97599999999999998}]},{"osd":7,"last update":"Fri Jul 14 11:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.66000000000000003,"15min":0.66200000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.72899999999999998,"15min":0.74299999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.84799999999999998,"5min":3.6440000000000001,"15min":3.6440000000000001},"last":1.0369999999999999}]},{"osd":8,"last update":"Fri Jul 14 11:59:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57699999999999996,"15min":0.57099999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.84999999999999998,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.754},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57299999999999995,"15min":0.57599999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.73699999999999999,"5min":0.874,"15min":0.874},"last":1.0820000000000001}]},{"osd":9,"last update":"Fri Jul 14 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.79000000000000004,"15min":0.78400000000000003},"min":{"1min":0.59499999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":1.389},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.77600000000000002,"15min":0.77000000000000002},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.875}]},{"osd":10,"last update":"Fri Jul 14 11:59:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":1.1399999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":1.2949999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36166,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36166,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36166,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36166,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36166,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T12:00:17.413 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-14T12:00:17.414 INFO:tasks.cephadm:Setting up client nodes... 2023-07-14T12:00:17.414 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-07-14T12:00:18.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:17 smithi093 ceph-mon[128922]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T12:00:18.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:17 smithi093 ceph-mon[128922]: mds.? [v2:172.21.15.97:6836/3684187131,v1:172.21.15.97:6837/3684187131] up:active 2023-07-14T12:00:18.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:17 smithi093 ceph-mon[128922]: fsmap cephfs:5 7 up:standby 5 up:active 2023-07-14T12:00:18.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:17 smithi093 ceph-mon[128922]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-14T12:00:18.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:17 smithi097 ceph-mon[129313]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T12:00:18.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:17 smithi097 ceph-mon[129313]: mds.? [v2:172.21.15.97:6836/3684187131,v1:172.21.15.97:6837/3684187131] up:active 2023-07-14T12:00:18.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:17 smithi097 ceph-mon[129313]: fsmap cephfs:5 7 up:standby 5 up:active 2023-07-14T12:00:18.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:17 smithi097 ceph-mon[129313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-14T12:00:18.143 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:17 smithi017 ceph-mon[126339]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T12:00:18.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:17 smithi017 ceph-mon[126339]: mds.? [v2:172.21.15.97:6836/3684187131,v1:172.21.15.97:6837/3684187131] up:active 2023-07-14T12:00:18.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:17 smithi017 ceph-mon[126339]: fsmap cephfs:5 7 up:standby 5 up:active 2023-07-14T12:00:18.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:17 smithi017 ceph-mon[126339]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-14T12:00:18.179 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T12:00:19.079 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:18 smithi017 ceph-mon[126339]: from='client.24815 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T12:00:19.080 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:18 smithi017 ceph-mon[126339]: pgmap v296: 129 pgs: 129 active+clean; 584 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s wr, 8 op/s 2023-07-14T12:00:19.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:18 smithi093 ceph-mon[128922]: from='client.24815 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T12:00:19.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:18 smithi093 ceph-mon[128922]: pgmap v296: 129 pgs: 129 active+clean; 584 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s wr, 8 op/s 2023-07-14T12:00:19.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:18 smithi097 ceph-mon[129313]: from='client.24815 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T12:00:19.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:18 smithi097 ceph-mon[129313]: pgmap v296: 129 pgs: 129 active+clean; 584 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s wr, 8 op/s 2023-07-14T12:00:19.731 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:19.750+0000 7f1229bd3700 1 -- 172.21.15.17:0/2469196260 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12241058e0 msgr2=0x7f1224107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:19.731 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:19.750+0000 7f1229bd3700 1 --2- 172.21.15.17:0/2469196260 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12241058e0 0x7f1224107ce0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f121c00b0a0 tx=0x7f121c00b3b0 comp rx=0 tx=0).stop 2023-07-14T12:00:19.731 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:19.750+0000 7f1229bd3700 1 -- 172.21.15.17:0/2469196260 shutdown_connections 2023-07-14T12:00:19.732 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:19.750+0000 7f1229bd3700 1 --2- 172.21.15.17:0/2469196260 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f12241006a0 0x7f1224102a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:19.732 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:19.750+0000 7f1229bd3700 1 --2- 172.21.15.17:0/2469196260 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12241058e0 0x7f1224107ce0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:19.732 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:19.750+0000 7f1229bd3700 1 --2- 172.21.15.17:0/2469196260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1224102fc0 0x7f12241053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:19.732 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:19.750+0000 7f1229bd3700 1 -- 172.21.15.17:0/2469196260 >> 172.21.15.17:0/2469196260 conn(0x7f12240fa0e0 msgr2=0x7f12240fc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:19.732 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:19.750+0000 7f1229bd3700 1 -- 172.21.15.17:0/2469196260 shutdown_connections 2023-07-14T12:00:19.732 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:19.750+0000 7f1229bd3700 1 -- 172.21.15.17:0/2469196260 wait complete. 2023-07-14T12:00:19.733 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:19.751+0000 7f1229bd3700 1 Processor -- start 2023-07-14T12:00:19.733 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:19.752+0000 7f1229bd3700 1 -- start start 2023-07-14T12:00:19.734 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:19.752+0000 7f1229bd3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12241006a0 0x7f1224197c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:19.734 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:19.752+0000 7f1229bd3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1224102fc0 0x7f1224198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:19.734 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:19.752+0000 7f1229bd3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12241058e0 0x7f122419f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:19.734 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:19.752+0000 7f1229bd3700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122419f760 con 0x7f12241058e0 2023-07-14T12:00:19.734 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:19.752+0000 7f1229bd3700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122419f8a0 con 0x7f1224102fc0 2023-07-14T12:00:19.734 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:19.752+0000 7f1229bd3700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122419fa60 con 0x7f12241006a0 2023-07-14T12:00:19.734 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:19.752+0000 7f1222ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1224102fc0 0x7f1224198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:19.735 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:19.752+0000 7f1223fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12241058e0 0x7f122419f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:19.735 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:19.752+0000 7f1222ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1224102fc0 0x7f1224198150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.17:57676/0 (socket says 172.21.15.17:57676) 2023-07-14T12:00:19.735 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:19.752+0000 7f1222ffd700 1 -- 172.21.15.17:0/2686382846 learned_addr learned my addr 172.21.15.17:0/2686382846 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:00:19.735 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:19.752+0000 7f1223fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12241058e0 0x7f122419f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44982/0 (socket says 172.21.15.17:44982) 2023-07-14T12:00:19.735 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:19.752+0000 7f12237fe700 1 --2- 172.21.15.17:0/2686382846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12241006a0 0x7f1224197c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:19.735 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:19.753+0000 7f1222ffd700 1 -- 172.21.15.17:0/2686382846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12241006a0 msgr2=0x7f1224197c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:19.735 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:19.753+0000 7f1222ffd700 1 --2- 172.21.15.17:0/2686382846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12241006a0 0x7f1224197c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:19.735 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:19.753+0000 7f1222ffd700 1 -- 172.21.15.17:0/2686382846 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12241058e0 msgr2=0x7f122419f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:19.736 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:19.753+0000 7f1222ffd700 1 --2- 172.21.15.17:0/2686382846 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12241058e0 0x7f122419f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:19.736 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:19.753+0000 7f1222ffd700 1 -- 172.21.15.17:0/2686382846 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f121c009d00 con 0x7f1224102fc0 2023-07-14T12:00:19.736 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:19.753+0000 7f1223fff700 1 --2- 172.21.15.17:0/2686382846 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12241058e0 0x7f122419f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T12:00:19.736 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:19.753+0000 7f1222ffd700 1 --2- 172.21.15.17:0/2686382846 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1224102fc0 0x7f1224198150 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f121800e9a0 tx=0x7f121800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:19.737 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:19.753+0000 7f1220ff9700 1 -- 172.21.15.17:0/2686382846 <== mon.2 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1218009790 con 0x7f1224102fc0 2023-07-14T12:00:19.737 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:19.753+0000 7f1220ff9700 1 -- 172.21.15.17:0/2686382846 <== mon.2 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1218010430 con 0x7f1224102fc0 2023-07-14T12:00:19.737 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:19.753+0000 7f1229bd3700 1 -- 172.21.15.17:0/2686382846 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f122419fd40 con 0x7f1224102fc0 2023-07-14T12:00:19.737 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:19.753+0000 7f1220ff9700 1 -- 172.21.15.17:0/2686382846 <== mon.2 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1218018680 con 0x7f1224102fc0 2023-07-14T12:00:19.737 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:19.754+0000 7f1229bd3700 1 -- 172.21.15.17:0/2686382846 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12241a0190 con 0x7f1224102fc0 2023-07-14T12:00:19.738 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:19.754+0000 7f1229bd3700 1 -- 172.21.15.17:0/2686382846 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1224107450 con 0x7f1224102fc0 2023-07-14T12:00:19.738 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:19.757+0000 7f1220ff9700 1 -- 172.21.15.17:0/2686382846 <== mon.2 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f12180105a0 con 0x7f1224102fc0 2023-07-14T12:00:19.739 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:19.758+0000 7f1220ff9700 1 --2- 172.21.15.17:0/2686382846 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f120c0a1150 0x7f120c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:19.739 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:19.758+0000 7f1220ff9700 1 -- 172.21.15.17:0/2686382846 <== mon.2 v2:172.21.15.93:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1218014070 con 0x7f1224102fc0 2023-07-14T12:00:19.739 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:19.759+0000 7f12237fe700 1 --2- 172.21.15.17:0/2686382846 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f120c0a1150 0x7f120c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:19.739 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:19.759+0000 7f12237fe700 1 --2- 172.21.15.17:0/2686382846 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f120c0a1150 0x7f120c0a3600 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f12241990f0 tx=0x7f1214009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:19.740 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:19.760+0000 7f1220ff9700 1 -- 172.21.15.17:0/2686382846 <== mon.2 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1218096410 con 0x7f1224102fc0 2023-07-14T12:00:20.031 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:20.051+0000 7f1229bd3700 1 -- 172.21.15.17:0/2686382846 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f122410b3c0 con 0x7f1224102fc0 2023-07-14T12:00:20.037 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:20.057+0000 7f1220ff9700 1 -- 172.21.15.17:0/2686382846 <== mon.2 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7f12180260f0 con 0x7f1224102fc0 2023-07-14T12:00:20.038 INFO:teuthology.orchestra.run.smithi017.stdout:[client.0] 2023-07-14T12:00:20.038 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDUOLFkZT0uAxAAmUixmxYA/L+9o6kVE5Q3nw== 2023-07-14T12:00:20.040 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:20.060+0000 7f1229bd3700 1 -- 172.21.15.17:0/2686382846 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f120c0a1150 msgr2=0x7f120c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:20.041 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:20.060+0000 7f1229bd3700 1 --2- 172.21.15.17:0/2686382846 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f120c0a1150 0x7f120c0a3600 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f12241990f0 tx=0x7f1214009260 comp rx=0 tx=0).stop 2023-07-14T12:00:20.041 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:20.060+0000 7f1229bd3700 1 -- 172.21.15.17:0/2686382846 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1224102fc0 msgr2=0x7f1224198150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:20.041 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:20.060+0000 7f1229bd3700 1 --2- 172.21.15.17:0/2686382846 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1224102fc0 0x7f1224198150 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f121800e9a0 tx=0x7f121800ecb0 comp rx=0 tx=0).stop 2023-07-14T12:00:20.041 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:20.060+0000 7f1229bd3700 1 -- 172.21.15.17:0/2686382846 shutdown_connections 2023-07-14T12:00:20.041 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:20.060+0000 7f1229bd3700 1 --2- 172.21.15.17:0/2686382846 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f120c0a1150 0x7f120c0a3600 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:20.041 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:20.060+0000 7f1229bd3700 1 --2- 172.21.15.17:0/2686382846 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12241058e0 0x7f122419f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:20.041 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:20.060+0000 7f1229bd3700 1 --2- 172.21.15.17:0/2686382846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12241006a0 0x7f1224197c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:20.041 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:20.060+0000 7f1229bd3700 1 --2- 172.21.15.17:0/2686382846 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1224102fc0 0x7f1224198150 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:20.042 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:20.060+0000 7f1229bd3700 1 -- 172.21.15.17:0/2686382846 >> 172.21.15.17:0/2686382846 conn(0x7f12240fa0e0 msgr2=0x7f12240fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:20.042 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:20.060+0000 7f1229bd3700 1 -- 172.21.15.17:0/2686382846 shutdown_connections 2023-07-14T12:00:20.042 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:20.060+0000 7f1229bd3700 1 -- 172.21.15.17:0/2686382846 wait complete. 2023-07-14T12:00:20.483 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2023-07-14T12:00:20.483 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-07-14T12:00:20.483 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-07-14T12:00:20.586 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-07-14T12:00:20.586 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-07-14T12:00:20.587 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph mgr dump --format=json 2023-07-14T12:00:21.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:20 smithi093 ceph-mon[128922]: pgmap v297: 129 pgs: 129 active+clean; 585 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2023-07-14T12:00:21.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:20 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/2686382846' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-14T12:00:21.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:20 smithi093 ceph-mon[128922]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-14T12:00:21.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:20 smithi093 ceph-mon[128922]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-07-14T12:00:21.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:20 smithi097 ceph-mon[129313]: pgmap v297: 129 pgs: 129 active+clean; 585 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2023-07-14T12:00:21.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:20 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/2686382846' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-14T12:00:21.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:20 smithi097 ceph-mon[129313]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-14T12:00:21.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:20 smithi097 ceph-mon[129313]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-07-14T12:00:21.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:20 smithi017 ceph-mon[126339]: pgmap v297: 129 pgs: 129 active+clean; 585 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2023-07-14T12:00:21.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:20 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/2686382846' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-14T12:00:21.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:20 smithi017 ceph-mon[126339]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-14T12:00:21.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:20 smithi017 ceph-mon[126339]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-07-14T12:00:21.358 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T12:00:23.021 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.040+0000 7fde3d9c3700 1 -- 172.21.15.17:0/3154472053 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fde380ffe00 msgr2=0x7fde381001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:23.021 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.040+0000 7fde3d9c3700 1 --2- 172.21.15.17:0/3154472053 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fde380ffe00 0x7fde381001d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fde28009990 tx=0x7fde28009ca0 comp rx=0 tx=0).stop 2023-07-14T12:00:23.022 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.040+0000 7fde3d9c3700 1 -- 172.21.15.17:0/3154472053 shutdown_connections 2023-07-14T12:00:23.022 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.040+0000 7fde3d9c3700 1 --2- 172.21.15.17:0/3154472053 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fde380ffe00 0x7fde381001d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:23.022 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.040+0000 7fde3d9c3700 1 --2- 172.21.15.17:0/3154472053 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde38104ea0 0x7fde3810ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:23.022 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.040+0000 7fde3d9c3700 1 --2- 172.21.15.17:0/3154472053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde381007a0 0x7fde38104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:23.022 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.040+0000 7fde3d9c3700 1 -- 172.21.15.17:0/3154472053 >> 172.21.15.17:0/3154472053 conn(0x7fde380fb660 msgr2=0x7fde380fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:23.022 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.040+0000 7fde3d9c3700 1 -- 172.21.15.17:0/3154472053 shutdown_connections 2023-07-14T12:00:23.022 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.041+0000 7fde3d9c3700 1 -- 172.21.15.17:0/3154472053 wait complete. 2023-07-14T12:00:23.023 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.042+0000 7fde3d9c3700 1 Processor -- start 2023-07-14T12:00:23.023 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.042+0000 7fde3d9c3700 1 -- start start 2023-07-14T12:00:23.023 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.043+0000 7fde3d9c3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde380ffe00 0x7fde38195cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:23.023 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.043+0000 7fde3d9c3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fde381007a0 0x7fde38196200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:23.024 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.043+0000 7fde3d9c3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde38104ea0 0x7fde381995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:23.025 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.043+0000 7fde3d9c3700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde38199bc0 con 0x7fde380ffe00 2023-07-14T12:00:23.025 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.043+0000 7fde3d9c3700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde38199d30 con 0x7fde381007a0 2023-07-14T12:00:23.025 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.043+0000 7fde3d9c3700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde38199ef0 con 0x7fde38104ea0 2023-07-14T12:00:23.025 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.043+0000 7fde36ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde380ffe00 0x7fde38195cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:23.025 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.043+0000 7fde36ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde380ffe00 0x7fde38195cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45856/0 (socket says 172.21.15.17:45856) 2023-07-14T12:00:23.025 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.043+0000 7fde36ffd700 1 -- 172.21.15.17:0/1493525413 learned_addr learned my addr 172.21.15.17:0/1493525413 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:00:23.025 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.043+0000 7fde367fc700 1 --2- 172.21.15.17:0/1493525413 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fde381007a0 0x7fde38196200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:23.026 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.043+0000 7fde377fe700 1 --2- 172.21.15.17:0/1493525413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde38104ea0 0x7fde381995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:23.026 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.044+0000 7fde36ffd700 1 -- 172.21.15.17:0/1493525413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde38104ea0 msgr2=0x7fde381995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:23.026 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.044+0000 7fde36ffd700 1 --2- 172.21.15.17:0/1493525413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde38104ea0 0x7fde381995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:23.026 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.044+0000 7fde36ffd700 1 -- 172.21.15.17:0/1493525413 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fde381007a0 msgr2=0x7fde38196200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:23.026 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.044+0000 7fde36ffd700 1 --2- 172.21.15.17:0/1493525413 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fde381007a0 0x7fde38196200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:23.026 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.044+0000 7fde36ffd700 1 -- 172.21.15.17:0/1493525413 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde280095f0 con 0x7fde380ffe00 2023-07-14T12:00:23.026 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.044+0000 7fde36ffd700 1 --2- 172.21.15.17:0/1493525413 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde380ffe00 0x7fde38195cc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fde28005910 tx=0x7fde28004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:23.026 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.044+0000 7fde3c9c1700 1 -- 172.21.15.17:0/1493525413 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2801d070 con 0x7fde380ffe00 2023-07-14T12:00:23.027 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.044+0000 7fde3d9c3700 1 -- 172.21.15.17:0/1493525413 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde3819a170 con 0x7fde380ffe00 2023-07-14T12:00:23.027 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.044+0000 7fde3c9c1700 1 -- 172.21.15.17:0/1493525413 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde28022470 con 0x7fde380ffe00 2023-07-14T12:00:23.027 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.044+0000 7fde3c9c1700 1 -- 172.21.15.17:0/1493525413 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde28017650 con 0x7fde380ffe00 2023-07-14T12:00:23.028 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.044+0000 7fde3d9c3700 1 -- 172.21.15.17:0/1493525413 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde381a2b20 con 0x7fde380ffe00 2023-07-14T12:00:23.028 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.045+0000 7fde3d9c3700 1 -- 172.21.15.17:0/1493525413 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde3804e210 con 0x7fde380ffe00 2023-07-14T12:00:23.028 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.047+0000 7fde3c9c1700 1 -- 172.21.15.17:0/1493525413 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fde280225e0 con 0x7fde380ffe00 2023-07-14T12:00:23.029 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.049+0000 7fde3c9c1700 1 --2- 172.21.15.17:0/1493525413 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fde200a11d0 0x7fde200a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:23.030 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.049+0000 7fde3c9c1700 1 -- 172.21.15.17:0/1493525413 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fde280cde00 con 0x7fde380ffe00 2023-07-14T12:00:23.030 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.049+0000 7fde367fc700 1 --2- 172.21.15.17:0/1493525413 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fde200a11d0 0x7fde200a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:23.030 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.050+0000 7fde367fc700 1 --2- 172.21.15.17:0/1493525413 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fde200a11d0 0x7fde200a3680 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fde381972a0 tx=0x7fde2c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:23.033 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.053+0000 7fde3c9c1700 1 -- 172.21.15.17:0/1493525413 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fde2805c790 con 0x7fde380ffe00 2023-07-14T12:00:23.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:22 smithi093 ceph-mon[128922]: pgmap v298: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 4.0 KiB/s wr, 16 op/s 2023-07-14T12:00:23.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:22 smithi097 ceph-mon[129313]: pgmap v298: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 4.0 KiB/s wr, 16 op/s 2023-07-14T12:00:23.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:22 smithi017 ceph-mon[126339]: pgmap v298: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 4.0 KiB/s wr, 16 op/s 2023-07-14T12:00:23.298 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.317+0000 7fde3d9c3700 1 -- 172.21.15.17:0/1493525413 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fde38196fe0 con 0x7fde380ffe00 2023-07-14T12:00:23.303 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.323+0000 7fde3c9c1700 1 -- 172.21.15.17:0/1493525413 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v16) v1 ==== 74+0+260034 (secure 0 0 0) 0x7fde280707e0 con 0x7fde380ffe00 2023-07-14T12:00:23.307 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T12:00:23.309 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.327+0000 7fde3d9c3700 1 -- 172.21.15.17:0/1493525413 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fde200a11d0 msgr2=0x7fde200a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:23.309 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.327+0000 7fde3d9c3700 1 --2- 172.21.15.17:0/1493525413 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fde200a11d0 0x7fde200a3680 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fde381972a0 tx=0x7fde2c009260 comp rx=0 tx=0).stop 2023-07-14T12:00:23.309 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.327+0000 7fde3d9c3700 1 -- 172.21.15.17:0/1493525413 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde380ffe00 msgr2=0x7fde38195cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:23.309 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.327+0000 7fde3d9c3700 1 --2- 172.21.15.17:0/1493525413 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde380ffe00 0x7fde38195cc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fde28005910 tx=0x7fde28004710 comp rx=0 tx=0).stop 2023-07-14T12:00:23.309 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.327+0000 7fde3d9c3700 1 -- 172.21.15.17:0/1493525413 shutdown_connections 2023-07-14T12:00:23.309 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.327+0000 7fde3d9c3700 1 --2- 172.21.15.17:0/1493525413 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fde200a11d0 0x7fde200a3680 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:23.310 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.327+0000 7fde3d9c3700 1 --2- 172.21.15.17:0/1493525413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde38104ea0 0x7fde381995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:23.310 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.327+0000 7fde3d9c3700 1 --2- 172.21.15.17:0/1493525413 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde380ffe00 0x7fde38195cc0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:23.310 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.327+0000 7fde3d9c3700 1 --2- 172.21.15.17:0/1493525413 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fde381007a0 0x7fde38196200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:23.310 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.327+0000 7fde3d9c3700 1 -- 172.21.15.17:0/1493525413 >> 172.21.15.17:0/1493525413 conn(0x7fde380fb660 msgr2=0x7fde380fc180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:23.310 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.328+0000 7fde3d9c3700 1 -- 172.21.15.17:0/1493525413 shutdown_connections 2023-07-14T12:00:23.310 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:23.328+0000 7fde3d9c3700 1 -- 172.21.15.17:0/1493525413 wait complete. 2023-07-14T12:00:23.856 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":16,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":4237449858},{"type":"v1","addr":"172.21.15.17:6801","nonce":4237449858}]},"active_addr":"172.21.15.17:6801/4237449858","active_change":"2023-07-14T11:53:05.639702+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.17:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.17:0","nonce":896985943}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.17:0","nonce":3180525686}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.17:0","nonce":915503706}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.17:0","nonce":28769718}]}]} 2023-07-14T12:00:23.863 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-07-14T12:00:23.863 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-07-14T12:00:23.863 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph osd dump --format=json 2023-07-14T12:00:24.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:23 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/1493525413' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-07-14T12:00:24.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:23 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/1493525413' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-07-14T12:00:24.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:23 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/1493525413' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-07-14T12:00:24.597 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T12:00:25.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:24 smithi093 ceph-mon[128922]: pgmap v299: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.0 KiB/s wr, 20 op/s 2023-07-14T12:00:25.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:24 smithi097 ceph-mon[129313]: pgmap v299: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.0 KiB/s wr, 20 op/s 2023-07-14T12:00:25.143 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:24 smithi017 ceph-mon[126339]: pgmap v299: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.0 KiB/s wr, 20 op/s 2023-07-14T12:00:26.429 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.448+0000 7fdf03a83700 1 -- 172.21.15.17:0/3601141214 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdefc104ea0 msgr2=0x7fdefc10ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:26.429 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.448+0000 7fdf03a83700 1 --2- 172.21.15.17:0/3601141214 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdefc104ea0 0x7fdefc10ca20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdef400b0d0 tx=0x7fdef400b3e0 comp rx=0 tx=0).stop 2023-07-14T12:00:26.429 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.448+0000 7fdf03a83700 1 -- 172.21.15.17:0/3601141214 shutdown_connections 2023-07-14T12:00:26.430 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.448+0000 7fdf03a83700 1 --2- 172.21.15.17:0/3601141214 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdefc0ffe00 0x7fdefc1001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:26.430 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.448+0000 7fdf03a83700 1 --2- 172.21.15.17:0/3601141214 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdefc104ea0 0x7fdefc10ca20 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:26.430 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.448+0000 7fdf03a83700 1 --2- 172.21.15.17:0/3601141214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdefc1007a0 0x7fdefc104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:26.430 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.448+0000 7fdf03a83700 1 -- 172.21.15.17:0/3601141214 >> 172.21.15.17:0/3601141214 conn(0x7fdefc0fb660 msgr2=0x7fdefc0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:26.430 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.448+0000 7fdf03a83700 1 -- 172.21.15.17:0/3601141214 shutdown_connections 2023-07-14T12:00:26.430 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.448+0000 7fdf03a83700 1 -- 172.21.15.17:0/3601141214 wait complete. 2023-07-14T12:00:26.432 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.449+0000 7fdf03a83700 1 Processor -- start 2023-07-14T12:00:26.432 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.449+0000 7fdf03a83700 1 -- start start 2023-07-14T12:00:26.432 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.450+0000 7fdf03a83700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdefc0ffe00 0x7fdefc195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:26.432 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.450+0000 7fdf03a83700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdefc1007a0 0x7fdefc1962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:26.432 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.450+0000 7fdf03a83700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdefc104ea0 0x7fdefc19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:26.432 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.450+0000 7fdf03a83700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc19ad20 con 0x7fdefc104ea0 2023-07-14T12:00:26.433 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.450+0000 7fdf03a83700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc19ae90 con 0x7fdefc1007a0 2023-07-14T12:00:26.433 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.450+0000 7fdf03a83700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc19b050 con 0x7fdefc0ffe00 2023-07-14T12:00:26.433 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.450+0000 7fdf0181f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdefc0ffe00 0x7fdefc195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:26.433 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.450+0000 7fdf0181f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdefc0ffe00 0x7fdefc195d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.17:50132/0 (socket says 172.21.15.17:50132) 2023-07-14T12:00:26.433 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.450+0000 7fdf0181f700 1 -- 172.21.15.17:0/1542386930 learned_addr learned my addr 172.21.15.17:0/1542386930 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:00:26.433 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.450+0000 7fdf02020700 1 --2- 172.21.15.17:0/1542386930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdefc104ea0 0x7fdefc19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:26.433 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.450+0000 7fdf0101e700 1 --2- 172.21.15.17:0/1542386930 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdefc1007a0 0x7fdefc1962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:26.433 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.450+0000 7fdf0181f700 1 -- 172.21.15.17:0/1542386930 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdefc1007a0 msgr2=0x7fdefc1962a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:26.434 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.450+0000 7fdf0181f700 1 --2- 172.21.15.17:0/1542386930 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdefc1007a0 0x7fdefc1962a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:26.434 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.450+0000 7fdf0181f700 1 -- 172.21.15.17:0/1542386930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdefc104ea0 msgr2=0x7fdefc19a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:26.434 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.450+0000 7fdf0181f700 1 --2- 172.21.15.17:0/1542386930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdefc104ea0 0x7fdefc19a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:26.434 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.450+0000 7fdf0181f700 1 -- 172.21.15.17:0/1542386930 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef4009d00 con 0x7fdefc0ffe00 2023-07-14T12:00:26.434 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.450+0000 7fdf0101e700 1 --2- 172.21.15.17:0/1542386930 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdefc1007a0 0x7fdefc1962a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T12:00:26.434 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.450+0000 7fdf02020700 1 --2- 172.21.15.17:0/1542386930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdefc104ea0 0x7fdefc19a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T12:00:26.434 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.451+0000 7fdf0181f700 1 --2- 172.21.15.17:0/1542386930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdefc0ffe00 0x7fdefc195d60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdef800d980 tx=0x7fdef800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:26.435 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.451+0000 7fdef2ffd700 1 -- 172.21.15.17:0/1542386930 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef8004220 con 0x7fdefc0ffe00 2023-07-14T12:00:26.435 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.451+0000 7fdf03a83700 1 -- 172.21.15.17:0/1542386930 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdefc19b330 con 0x7fdefc0ffe00 2023-07-14T12:00:26.435 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.451+0000 7fdef2ffd700 1 -- 172.21.15.17:0/1542386930 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdef8007640 con 0x7fdefc0ffe00 2023-07-14T12:00:26.435 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.451+0000 7fdef2ffd700 1 -- 172.21.15.17:0/1542386930 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef8005610 con 0x7fdefc0ffe00 2023-07-14T12:00:26.435 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.451+0000 7fdf03a83700 1 -- 172.21.15.17:0/1542386930 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdefc1a2b90 con 0x7fdefc0ffe00 2023-07-14T12:00:26.435 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.452+0000 7fdf03a83700 1 -- 172.21.15.17:0/1542386930 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdefc04e210 con 0x7fdefc0ffe00 2023-07-14T12:00:26.436 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.454+0000 7fdef2ffd700 1 -- 172.21.15.17:0/1542386930 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fdef8004750 con 0x7fdefc0ffe00 2023-07-14T12:00:26.436 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.455+0000 7fdef2ffd700 1 --2- 172.21.15.17:0/1542386930 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fdee80a11d0 0x7fdee80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:26.436 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.456+0000 7fdef2ffd700 1 -- 172.21.15.17:0/1542386930 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fdef80cccf0 con 0x7fdefc0ffe00 2023-07-14T12:00:26.437 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.456+0000 7fdf0101e700 1 --2- 172.21.15.17:0/1542386930 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fdee80a11d0 0x7fdee80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:26.437 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.456+0000 7fdf0101e700 1 --2- 172.21.15.17:0/1542386930 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fdee80a11d0 0x7fdee80a3680 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdefc197380 tx=0x7fdeec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:26.438 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.458+0000 7fdef2ffd700 1 -- 172.21.15.17:0/1542386930 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fdef80960d0 con 0x7fdefc0ffe00 2023-07-14T12:00:26.646 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.666+0000 7fdf03a83700 1 -- 172.21.15.17:0/1542386930 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdefc1970c0 con 0x7fdefc0ffe00 2023-07-14T12:00:26.648 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.668+0000 7fdef2ffd700 1 -- 172.21.15.17:0/1542386930 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24336 (secure 0 0 0) 0x7fdef804df20 con 0x7fdefc0ffe00 2023-07-14T12:00:26.649 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T12:00:26.649 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":90,"fsid":"b3c386cc-223c-11ee-9b33-001a4aab830c","created":"2023-07-14T11:51:59.992960+0000","modified":"2023-07-14T12:00:06.800621+0000","last_up_change":"2023-07-14T11:58:55.037769+0000","last_in_change":"2023-07-14T11:58:37.114870+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:55:28.237002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:59:41.560502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T11:59:42.725461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:59:46.267029+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"57003438-a83c-4c8e-a029-56414740895c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6803","nonce":3588366384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6805","nonce":3588366384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6809","nonce":3588366384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6807","nonce":3588366384}]},"public_addr":"172.21.15.17:6803/3588366384","cluster_addr":"172.21.15.17:6805/3588366384","heartbeat_back_addr":"172.21.15.17:6809/3588366384","heartbeat_front_addr":"172.21.15.17:6807/3588366384","state":["exists","up"]},{"osd":1,"uuid":"f56b6337-d8a9-4411-b9d8-b6863dc98556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6801","nonce":3147203064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6803","nonce":3147203064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6807","nonce":3147203064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6805","nonce":3147203064}]},"public_addr":"172.21.15.93:6801/3147203064","cluster_addr":"172.21.15.93:6803/3147203064","heartbeat_back_addr":"172.21.15.93:6807/3147203064","heartbeat_front_addr":"172.21.15.93:6805/3147203064","state":["exists","up"]},{"osd":2,"uuid":"fc34bb7f-551f-46e1-99c0-59924d6c1346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6801","nonce":1877269815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6803","nonce":1877269815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6807","nonce":1877269815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6805","nonce":1877269815}]},"public_addr":"172.21.15.97:6801/1877269815","cluster_addr":"172.21.15.97:6803/1877269815","heartbeat_back_addr":"172.21.15.97:6807/1877269815","heartbeat_front_addr":"172.21.15.97:6805/1877269815","state":["exists","up"]},{"osd":3,"uuid":"37a1583c-4fb8-412a-a0c8-f83b1ff26a91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6811","nonce":2784619821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6813","nonce":2784619821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6817","nonce":2784619821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6815","nonce":2784619821}]},"public_addr":"172.21.15.17:6811/2784619821","cluster_addr":"172.21.15.17:6813/2784619821","heartbeat_back_addr":"172.21.15.17:6817/2784619821","heartbeat_front_addr":"172.21.15.17:6815/2784619821","state":["exists","up"]},{"osd":4,"uuid":"f4b9f377-d519-4b14-ac4a-f1debaeed9f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6809","nonce":3151801173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6811","nonce":3151801173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6815","nonce":3151801173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6813","nonce":3151801173}]},"public_addr":"172.21.15.93:6809/3151801173","cluster_addr":"172.21.15.93:6811/3151801173","heartbeat_back_addr":"172.21.15.93:6815/3151801173","heartbeat_front_addr":"172.21.15.93:6813/3151801173","state":["exists","up"]},{"osd":5,"uuid":"2d2af354-0535-4ef9-b166-3ef219498db8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6809","nonce":2184413924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6811","nonce":2184413924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6815","nonce":2184413924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6813","nonce":2184413924}]},"public_addr":"172.21.15.97:6809/2184413924","cluster_addr":"172.21.15.97:6811/2184413924","heartbeat_back_addr":"172.21.15.97:6815/2184413924","heartbeat_front_addr":"172.21.15.97:6813/2184413924","state":["exists","up"]},{"osd":6,"uuid":"9cdf0c4b-62dc-4168-864e-a9d1650a9ea8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6819","nonce":50486581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6821","nonce":50486581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6825","nonce":50486581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6823","nonce":50486581}]},"public_addr":"172.21.15.17:6819/50486581","cluster_addr":"172.21.15.17:6821/50486581","heartbeat_back_addr":"172.21.15.17:6825/50486581","heartbeat_front_addr":"172.21.15.17:6823/50486581","state":["exists","up"]},{"osd":7,"uuid":"09f60823-5491-45cd-9324-6ae0d56ce262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6817","nonce":2596912392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6819","nonce":2596912392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6823","nonce":2596912392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6821","nonce":2596912392}]},"public_addr":"172.21.15.93:6817/2596912392","cluster_addr":"172.21.15.93:6819/2596912392","heartbeat_back_addr":"172.21.15.93:6823/2596912392","heartbeat_front_addr":"172.21.15.93:6821/2596912392","state":["exists","up"]},{"osd":8,"uuid":"8c5771c9-718e-4313-a2b6-5e1c37c53f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6817","nonce":1965492859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6819","nonce":1965492859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6823","nonce":1965492859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6821","nonce":1965492859}]},"public_addr":"172.21.15.97:6817/1965492859","cluster_addr":"172.21.15.97:6819/1965492859","heartbeat_back_addr":"172.21.15.97:6823/1965492859","heartbeat_front_addr":"172.21.15.97:6821/1965492859","state":["exists","up"]},{"osd":9,"uuid":"1197b8a3-c09f-4c57-aba1-9e988a1620b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6827","nonce":2512645700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6829","nonce":2512645700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6833","nonce":2512645700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6831","nonce":2512645700}]},"public_addr":"172.21.15.17:6827/2512645700","cluster_addr":"172.21.15.17:6829/2512645700","heartbeat_back_addr":"172.21.15.17:6833/2512645700","heartbeat_front_addr":"172.21.15.17:6831/2512645700","state":["exists","up"]},{"osd":10,"uuid":"f99918c2-cdd8-4a7a-be2e-d1d6e0f828a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6825","nonce":264429558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6827","nonce":264429558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6831","nonce":264429558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6829","nonce":264429558}]},"public_addr":"172.21.15.93:6825/264429558","cluster_addr":"172.21.15.93:6827/264429558","heartbeat_back_addr":"172.21.15.93:6831/264429558","heartbeat_front_addr":"172.21.15.93:6829/264429558","state":["exists","up"]},{"osd":11,"uuid":"6bbb5b79-8d4d-441f-9583-203df6c78cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6825","nonce":3389913754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6827","nonce":3389913754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6831","nonce":3389913754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6829","nonce":3389913754}]},"public_addr":"172.21.15.97:6825/3389913754","cluster_addr":"172.21.15.97:6827/3389913754","heartbeat_back_addr":"172.21.15.97:6831/3389913754","heartbeat_front_addr":"172.21.15.97:6829/3389913754","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:54:41.079618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:03.207899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:24.136961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:50.392609+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:56:12.136237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:56:33.212314+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:00.243646+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:22.059234+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:43.232399+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:10.043896+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:31.910780+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:53.158983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.9","mappings":[{"from":5,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/2084033276":"2023-07-15T11:53:05.639597+0000","172.21.15.17:6801/1059284716":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/3651785636":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/1373538024":"2023-07-15T11:52:37.887946+0000","172.21.15.17:0/3560395020":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6800/1059284716":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/4262912861":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/2258375202":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6800/2804227742":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6801/2804227742":"2023-07-15T11:52:37.887946+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T12:00:26.650 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.670+0000 7fdf03a83700 1 -- 172.21.15.17:0/1542386930 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fdee80a11d0 msgr2=0x7fdee80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:26.651 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.670+0000 7fdf03a83700 1 --2- 172.21.15.17:0/1542386930 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fdee80a11d0 0x7fdee80a3680 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdefc197380 tx=0x7fdeec009260 comp rx=0 tx=0).stop 2023-07-14T12:00:26.651 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.670+0000 7fdf03a83700 1 -- 172.21.15.17:0/1542386930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdefc0ffe00 msgr2=0x7fdefc195d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:26.651 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.670+0000 7fdf03a83700 1 --2- 172.21.15.17:0/1542386930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdefc0ffe00 0x7fdefc195d60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdef800d980 tx=0x7fdef800dc90 comp rx=0 tx=0).stop 2023-07-14T12:00:26.651 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.670+0000 7fdf03a83700 1 -- 172.21.15.17:0/1542386930 shutdown_connections 2023-07-14T12:00:26.651 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.670+0000 7fdf03a83700 1 --2- 172.21.15.17:0/1542386930 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fdee80a11d0 0x7fdee80a3680 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:26.651 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.670+0000 7fdf03a83700 1 --2- 172.21.15.17:0/1542386930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdefc104ea0 0x7fdefc19a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:26.651 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.670+0000 7fdf03a83700 1 --2- 172.21.15.17:0/1542386930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdefc0ffe00 0x7fdefc195d60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:26.652 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.670+0000 7fdf03a83700 1 --2- 172.21.15.17:0/1542386930 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdefc1007a0 0x7fdefc1962a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:26.652 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.670+0000 7fdf03a83700 1 -- 172.21.15.17:0/1542386930 >> 172.21.15.17:0/1542386930 conn(0x7fdefc0fb660 msgr2=0x7fdefc0fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:26.652 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.670+0000 7fdf03a83700 1 -- 172.21.15.17:0/1542386930 shutdown_connections 2023-07-14T12:00:26.652 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:26.670+0000 7fdf03a83700 1 -- 172.21.15.17:0/1542386930 wait complete. 2023-07-14T12:00:27.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:26 smithi093 ceph-mon[128922]: pgmap v300: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.9 KiB/s wr, 20 op/s 2023-07-14T12:00:27.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:26 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/1542386930' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T12:00:27.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:26 smithi097 ceph-mon[129313]: pgmap v300: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.9 KiB/s wr, 20 op/s 2023-07-14T12:00:27.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:26 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/1542386930' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T12:00:27.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:26 smithi017 ceph-mon[126339]: pgmap v300: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.9 KiB/s wr, 20 op/s 2023-07-14T12:00:27.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:26 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/1542386930' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T12:00:27.302 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-07-14T12:00:27.302 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph osd dump --format=json 2023-07-14T12:00:28.036 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T12:00:29.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:28 smithi093 ceph-mon[128922]: pgmap v301: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.0 KiB/s wr, 20 op/s 2023-07-14T12:00:29.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:28 smithi097 ceph-mon[129313]: pgmap v301: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.0 KiB/s wr, 20 op/s 2023-07-14T12:00:29.143 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:28 smithi017 ceph-mon[126339]: pgmap v301: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.0 KiB/s wr, 20 op/s 2023-07-14T12:00:29.671 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.690+0000 7feaefd2f700 1 -- 172.21.15.17:0/2263986188 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feae80ffe00 msgr2=0x7feae81001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:29.671 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.690+0000 7feaefd2f700 1 --2- 172.21.15.17:0/2263986188 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feae80ffe00 0x7feae81001d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7feae40099c0 tx=0x7feae4009cd0 comp rx=0 tx=0).stop 2023-07-14T12:00:29.671 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.690+0000 7feaefd2f700 1 -- 172.21.15.17:0/2263986188 shutdown_connections 2023-07-14T12:00:29.671 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.690+0000 7feaefd2f700 1 --2- 172.21.15.17:0/2263986188 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feae8104ea0 0x7feae810ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:29.672 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.690+0000 7feaefd2f700 1 --2- 172.21.15.17:0/2263986188 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feae80ffe00 0x7feae81001d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:29.672 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.690+0000 7feaefd2f700 1 --2- 172.21.15.17:0/2263986188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feae81007a0 0x7feae8104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:29.672 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.690+0000 7feaefd2f700 1 -- 172.21.15.17:0/2263986188 >> 172.21.15.17:0/2263986188 conn(0x7feae80fb660 msgr2=0x7feae80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:29.672 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.690+0000 7feaefd2f700 1 -- 172.21.15.17:0/2263986188 shutdown_connections 2023-07-14T12:00:29.672 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.690+0000 7feaefd2f700 1 -- 172.21.15.17:0/2263986188 wait complete. 2023-07-14T12:00:29.674 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.691+0000 7feaefd2f700 1 Processor -- start 2023-07-14T12:00:29.674 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.691+0000 7feaefd2f700 1 -- start start 2023-07-14T12:00:29.674 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.691+0000 7feaefd2f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feae80ffe00 0x7feae8072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:29.674 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.691+0000 7feaefd2f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feae81007a0 0x7feae8072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:29.674 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.692+0000 7feaefd2f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feae8104ea0 0x7feae80732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:29.674 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.692+0000 7feaefd2f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae81a4120 con 0x7feae8104ea0 2023-07-14T12:00:29.674 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.692+0000 7feaefd2f700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae81a4290 con 0x7feae80ffe00 2023-07-14T12:00:29.675 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.692+0000 7feaefd2f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae81a4450 con 0x7feae81007a0 2023-07-14T12:00:29.675 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.692+0000 7feaedacb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feae80ffe00 0x7feae8072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:29.675 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.692+0000 7feaed2ca700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feae81007a0 0x7feae8072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:29.675 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.692+0000 7feaee2cc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feae8104ea0 0x7feae80732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:29.675 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.692+0000 7feaedacb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feae80ffe00 0x7feae8072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.17:46082/0 (socket says 172.21.15.17:46082) 2023-07-14T12:00:29.675 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.692+0000 7feaedacb700 1 -- 172.21.15.17:0/1111611471 learned_addr learned my addr 172.21.15.17:0/1111611471 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:00:29.675 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.692+0000 7feaedacb700 1 -- 172.21.15.17:0/1111611471 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feae81007a0 msgr2=0x7feae8072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:29.675 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.692+0000 7feaedacb700 1 --2- 172.21.15.17:0/1111611471 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feae81007a0 0x7feae8072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:29.676 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.692+0000 7feaedacb700 1 -- 172.21.15.17:0/1111611471 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feae8104ea0 msgr2=0x7feae80732d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:29.676 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.692+0000 7feaedacb700 1 --2- 172.21.15.17:0/1111611471 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feae8104ea0 0x7feae80732d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:29.676 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.692+0000 7feaedacb700 1 -- 172.21.15.17:0/1111611471 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae40095f0 con 0x7feae80ffe00 2023-07-14T12:00:29.676 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.692+0000 7feaee2cc700 1 --2- 172.21.15.17:0/1111611471 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feae8104ea0 0x7feae80732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T12:00:29.676 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.693+0000 7feaedacb700 1 --2- 172.21.15.17:0/1111611471 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feae80ffe00 0x7feae8072810 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7feae40098e0 tx=0x7feae400f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:29.676 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.693+0000 7feadeffd700 1 -- 172.21.15.17:0/1111611471 <== mon.2 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae401d070 con 0x7feae80ffe00 2023-07-14T12:00:29.676 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.693+0000 7feadeffd700 1 -- 172.21.15.17:0/1111611471 <== mon.2 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feae4022470 con 0x7feae80ffe00 2023-07-14T12:00:29.677 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.693+0000 7feaefd2f700 1 -- 172.21.15.17:0/1111611471 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feae81a46d0 con 0x7feae80ffe00 2023-07-14T12:00:29.677 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.693+0000 7feadeffd700 1 -- 172.21.15.17:0/1111611471 <== mon.2 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae4017650 con 0x7feae80ffe00 2023-07-14T12:00:29.677 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.693+0000 7feaefd2f700 1 -- 172.21.15.17:0/1111611471 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feae81a4ae0 con 0x7feae80ffe00 2023-07-14T12:00:29.677 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.694+0000 7feaefd2f700 1 -- 172.21.15.17:0/1111611471 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feae804e210 con 0x7feae80ffe00 2023-07-14T12:00:29.677 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.697+0000 7feadeffd700 1 -- 172.21.15.17:0/1111611471 <== mon.2 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7feae4022850 con 0x7feae80ffe00 2023-07-14T12:00:29.678 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.698+0000 7feadeffd700 1 --2- 172.21.15.17:0/1111611471 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fead40a1150 0x7fead40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:29.679 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.698+0000 7feadeffd700 1 -- 172.21.15.17:0/1111611471 <== mon.2 v2:172.21.15.93:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7feae40cdd40 con 0x7feae80ffe00 2023-07-14T12:00:29.679 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.698+0000 7feaed2ca700 1 --2- 172.21.15.17:0/1111611471 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fead40a1150 0x7fead40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:29.679 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.699+0000 7feaed2ca700 1 --2- 172.21.15.17:0/1111611471 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fead40a1150 0x7fead40a3600 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7feae8074000 tx=0x7fead8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:29.680 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.700+0000 7feadeffd700 1 -- 172.21.15.17:0/1111611471 <== mon.2 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7feae40971b0 con 0x7feae80ffe00 2023-07-14T12:00:29.890 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.909+0000 7feaefd2f700 1 -- 172.21.15.17:0/1111611471 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feae8073d40 con 0x7feae80ffe00 2023-07-14T12:00:29.892 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.911+0000 7feadeffd700 1 -- 172.21.15.17:0/1111611471 <== mon.2 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24336 (secure 0 0 0) 0x7feae409a7d0 con 0x7feae80ffe00 2023-07-14T12:00:29.892 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T12:00:29.893 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":90,"fsid":"b3c386cc-223c-11ee-9b33-001a4aab830c","created":"2023-07-14T11:51:59.992960+0000","modified":"2023-07-14T12:00:06.800621+0000","last_up_change":"2023-07-14T11:58:55.037769+0000","last_in_change":"2023-07-14T11:58:37.114870+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:55:28.237002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:59:41.560502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T11:59:42.725461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:59:46.267029+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"57003438-a83c-4c8e-a029-56414740895c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6803","nonce":3588366384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6805","nonce":3588366384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6809","nonce":3588366384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6807","nonce":3588366384}]},"public_addr":"172.21.15.17:6803/3588366384","cluster_addr":"172.21.15.17:6805/3588366384","heartbeat_back_addr":"172.21.15.17:6809/3588366384","heartbeat_front_addr":"172.21.15.17:6807/3588366384","state":["exists","up"]},{"osd":1,"uuid":"f56b6337-d8a9-4411-b9d8-b6863dc98556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6801","nonce":3147203064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6803","nonce":3147203064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6807","nonce":3147203064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6805","nonce":3147203064}]},"public_addr":"172.21.15.93:6801/3147203064","cluster_addr":"172.21.15.93:6803/3147203064","heartbeat_back_addr":"172.21.15.93:6807/3147203064","heartbeat_front_addr":"172.21.15.93:6805/3147203064","state":["exists","up"]},{"osd":2,"uuid":"fc34bb7f-551f-46e1-99c0-59924d6c1346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6801","nonce":1877269815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6803","nonce":1877269815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6807","nonce":1877269815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6805","nonce":1877269815}]},"public_addr":"172.21.15.97:6801/1877269815","cluster_addr":"172.21.15.97:6803/1877269815","heartbeat_back_addr":"172.21.15.97:6807/1877269815","heartbeat_front_addr":"172.21.15.97:6805/1877269815","state":["exists","up"]},{"osd":3,"uuid":"37a1583c-4fb8-412a-a0c8-f83b1ff26a91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6811","nonce":2784619821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6813","nonce":2784619821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6817","nonce":2784619821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6815","nonce":2784619821}]},"public_addr":"172.21.15.17:6811/2784619821","cluster_addr":"172.21.15.17:6813/2784619821","heartbeat_back_addr":"172.21.15.17:6817/2784619821","heartbeat_front_addr":"172.21.15.17:6815/2784619821","state":["exists","up"]},{"osd":4,"uuid":"f4b9f377-d519-4b14-ac4a-f1debaeed9f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6809","nonce":3151801173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6811","nonce":3151801173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6815","nonce":3151801173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6813","nonce":3151801173}]},"public_addr":"172.21.15.93:6809/3151801173","cluster_addr":"172.21.15.93:6811/3151801173","heartbeat_back_addr":"172.21.15.93:6815/3151801173","heartbeat_front_addr":"172.21.15.93:6813/3151801173","state":["exists","up"]},{"osd":5,"uuid":"2d2af354-0535-4ef9-b166-3ef219498db8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6809","nonce":2184413924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6811","nonce":2184413924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6815","nonce":2184413924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6813","nonce":2184413924}]},"public_addr":"172.21.15.97:6809/2184413924","cluster_addr":"172.21.15.97:6811/2184413924","heartbeat_back_addr":"172.21.15.97:6815/2184413924","heartbeat_front_addr":"172.21.15.97:6813/2184413924","state":["exists","up"]},{"osd":6,"uuid":"9cdf0c4b-62dc-4168-864e-a9d1650a9ea8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6819","nonce":50486581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6821","nonce":50486581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6825","nonce":50486581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6823","nonce":50486581}]},"public_addr":"172.21.15.17:6819/50486581","cluster_addr":"172.21.15.17:6821/50486581","heartbeat_back_addr":"172.21.15.17:6825/50486581","heartbeat_front_addr":"172.21.15.17:6823/50486581","state":["exists","up"]},{"osd":7,"uuid":"09f60823-5491-45cd-9324-6ae0d56ce262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6817","nonce":2596912392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6819","nonce":2596912392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6823","nonce":2596912392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6821","nonce":2596912392}]},"public_addr":"172.21.15.93:6817/2596912392","cluster_addr":"172.21.15.93:6819/2596912392","heartbeat_back_addr":"172.21.15.93:6823/2596912392","heartbeat_front_addr":"172.21.15.93:6821/2596912392","state":["exists","up"]},{"osd":8,"uuid":"8c5771c9-718e-4313-a2b6-5e1c37c53f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6817","nonce":1965492859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6819","nonce":1965492859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6823","nonce":1965492859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6821","nonce":1965492859}]},"public_addr":"172.21.15.97:6817/1965492859","cluster_addr":"172.21.15.97:6819/1965492859","heartbeat_back_addr":"172.21.15.97:6823/1965492859","heartbeat_front_addr":"172.21.15.97:6821/1965492859","state":["exists","up"]},{"osd":9,"uuid":"1197b8a3-c09f-4c57-aba1-9e988a1620b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6827","nonce":2512645700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6829","nonce":2512645700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6833","nonce":2512645700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6831","nonce":2512645700}]},"public_addr":"172.21.15.17:6827/2512645700","cluster_addr":"172.21.15.17:6829/2512645700","heartbeat_back_addr":"172.21.15.17:6833/2512645700","heartbeat_front_addr":"172.21.15.17:6831/2512645700","state":["exists","up"]},{"osd":10,"uuid":"f99918c2-cdd8-4a7a-be2e-d1d6e0f828a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6825","nonce":264429558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6827","nonce":264429558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6831","nonce":264429558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6829","nonce":264429558}]},"public_addr":"172.21.15.93:6825/264429558","cluster_addr":"172.21.15.93:6827/264429558","heartbeat_back_addr":"172.21.15.93:6831/264429558","heartbeat_front_addr":"172.21.15.93:6829/264429558","state":["exists","up"]},{"osd":11,"uuid":"6bbb5b79-8d4d-441f-9583-203df6c78cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6825","nonce":3389913754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6827","nonce":3389913754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6831","nonce":3389913754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6829","nonce":3389913754}]},"public_addr":"172.21.15.97:6825/3389913754","cluster_addr":"172.21.15.97:6827/3389913754","heartbeat_back_addr":"172.21.15.97:6831/3389913754","heartbeat_front_addr":"172.21.15.97:6829/3389913754","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:54:41.079618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:03.207899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:24.136961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:50.392609+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:56:12.136237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:56:33.212314+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:00.243646+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:22.059234+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:43.232399+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:10.043896+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:31.910780+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:53.158983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.9","mappings":[{"from":5,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/2084033276":"2023-07-15T11:53:05.639597+0000","172.21.15.17:6801/1059284716":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/3651785636":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/1373538024":"2023-07-15T11:52:37.887946+0000","172.21.15.17:0/3560395020":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6800/1059284716":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/4262912861":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/2258375202":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6800/2804227742":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6801/2804227742":"2023-07-15T11:52:37.887946+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T12:00:29.894 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.914+0000 7feaefd2f700 1 -- 172.21.15.17:0/1111611471 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fead40a1150 msgr2=0x7fead40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:29.895 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.914+0000 7feaefd2f700 1 --2- 172.21.15.17:0/1111611471 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fead40a1150 0x7fead40a3600 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7feae8074000 tx=0x7fead8009260 comp rx=0 tx=0).stop 2023-07-14T12:00:29.895 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.914+0000 7feaefd2f700 1 -- 172.21.15.17:0/1111611471 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feae80ffe00 msgr2=0x7feae8072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:29.895 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.914+0000 7feaefd2f700 1 --2- 172.21.15.17:0/1111611471 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feae80ffe00 0x7feae8072810 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7feae40098e0 tx=0x7feae400f690 comp rx=0 tx=0).stop 2023-07-14T12:00:29.895 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.914+0000 7feaefd2f700 1 -- 172.21.15.17:0/1111611471 shutdown_connections 2023-07-14T12:00:29.895 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.914+0000 7feaefd2f700 1 --2- 172.21.15.17:0/1111611471 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fead40a1150 0x7fead40a3600 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:29.895 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.914+0000 7feaefd2f700 1 --2- 172.21.15.17:0/1111611471 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feae80ffe00 0x7feae8072810 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:29.896 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.914+0000 7feaefd2f700 1 --2- 172.21.15.17:0/1111611471 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feae8104ea0 0x7feae80732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:29.896 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.914+0000 7feaefd2f700 1 --2- 172.21.15.17:0/1111611471 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feae81007a0 0x7feae8072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:29.896 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.914+0000 7feaefd2f700 1 -- 172.21.15.17:0/1111611471 >> 172.21.15.17:0/1111611471 conn(0x7feae80fb660 msgr2=0x7feae80fc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:29.896 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.914+0000 7feaefd2f700 1 -- 172.21.15.17:0/1111611471 shutdown_connections 2023-07-14T12:00:29.896 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:29.914+0000 7feaefd2f700 1 -- 172.21.15.17:0/1111611471 wait complete. 2023-07-14T12:00:30.433 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph tell osd.0 flush_pg_stats 2023-07-14T12:00:30.434 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph tell osd.1 flush_pg_stats 2023-07-14T12:00:30.435 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph tell osd.2 flush_pg_stats 2023-07-14T12:00:30.435 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph tell osd.3 flush_pg_stats 2023-07-14T12:00:30.436 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph tell osd.4 flush_pg_stats 2023-07-14T12:00:30.437 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph tell osd.5 flush_pg_stats 2023-07-14T12:00:30.437 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph tell osd.6 flush_pg_stats 2023-07-14T12:00:30.438 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph tell osd.7 flush_pg_stats 2023-07-14T12:00:30.439 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph tell osd.8 flush_pg_stats 2023-07-14T12:00:30.439 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph tell osd.9 flush_pg_stats 2023-07-14T12:00:30.440 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph tell osd.10 flush_pg_stats 2023-07-14T12:00:30.440 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph tell osd.11 flush_pg_stats 2023-07-14T12:00:31.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:30 smithi093 ceph-mon[128922]: pgmap v302: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.2 KiB/s wr, 17 op/s 2023-07-14T12:00:31.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:30 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/1111611471' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T12:00:31.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:30 smithi097 ceph-mon[129313]: pgmap v302: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.2 KiB/s wr, 17 op/s 2023-07-14T12:00:31.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:30 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/1111611471' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T12:00:31.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:30 smithi017 ceph-mon[126339]: pgmap v302: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.2 KiB/s wr, 17 op/s 2023-07-14T12:00:31.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:30 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/1111611471' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T12:00:32.262 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T12:00:32.321 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T12:00:32.415 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T12:00:32.733 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T12:00:32.760 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T12:00:32.922 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:32 smithi017 ceph-mon[126339]: pgmap v303: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 853 B/s wr, 11 op/s 2023-07-14T12:00:33.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:32 smithi093 ceph-mon[128922]: pgmap v303: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 853 B/s wr, 11 op/s 2023-07-14T12:00:33.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:32 smithi097 ceph-mon[129313]: pgmap v303: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 853 B/s wr, 11 op/s 2023-07-14T12:00:33.734 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T12:00:33.908 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T12:00:34.164 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T12:00:34.239 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T12:00:34.423 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T12:00:34.471 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T12:00:34.539 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:34 smithi017 ceph-mon[126339]: pgmap v304: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 0 B/s wr, 12 op/s 2023-07-14T12:00:34.861 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:34 smithi093 ceph-mon[128922]: pgmap v304: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 0 B/s wr, 12 op/s 2023-07-14T12:00:34.884 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:34 smithi097 ceph-mon[129313]: pgmap v304: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 0 B/s wr, 12 op/s 2023-07-14T12:00:35.845 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T12:00:37.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:36 smithi093 ceph-mon[128922]: pgmap v305: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-14T12:00:37.126 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:36 smithi017 ceph-mon[126339]: pgmap v305: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-14T12:00:37.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:36 smithi097 ceph-mon[129313]: pgmap v305: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-14T12:00:38.393 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.412+0000 7f0f714bf700 1 -- 172.21.15.17:0/1677562307 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f6c0ff290 msgr2=0x7f0f6c10bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:38.393 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.412+0000 7f0f714bf700 1 --2- 172.21.15.17:0/1677562307 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f6c0ff290 0x7f0f6c10bff0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0f60009990 tx=0x7f0f60009ca0 comp rx=0 tx=0).stop 2023-07-14T12:00:38.393 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.412+0000 7f0f714bf700 1 -- 172.21.15.17:0/1677562307 shutdown_connections 2023-07-14T12:00:38.393 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.412+0000 7f0f714bf700 1 --2- 172.21.15.17:0/1677562307 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f6c10c530 0x7f0f6c10e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:38.393 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.412+0000 7f0f714bf700 1 --2- 172.21.15.17:0/1677562307 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f6c0fe980 0x7f0f6c0fed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:38.394 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.412+0000 7f0f714bf700 1 --2- 172.21.15.17:0/1677562307 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f6c0ff290 0x7f0f6c10bff0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:38.394 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.412+0000 7f0f714bf700 1 -- 172.21.15.17:0/1677562307 >> 172.21.15.17:0/1677562307 conn(0x7f0f6c075b70 msgr2=0x7f0f6c075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:38.394 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.412+0000 7f0f714bf700 1 -- 172.21.15.17:0/1677562307 shutdown_connections 2023-07-14T12:00:38.394 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.412+0000 7f0f714bf700 1 -- 172.21.15.17:0/1677562307 wait complete. 2023-07-14T12:00:38.395 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.413+0000 7f0f714bf700 1 Processor -- start 2023-07-14T12:00:38.395 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.413+0000 7f0f714bf700 1 -- start start 2023-07-14T12:00:38.396 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.414+0000 7f0f714bf700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f6c0fe980 0x7f0f6c197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:38.396 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.414+0000 7f0f714bf700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f6c0ff290 0x7f0f6c198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:38.396 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.414+0000 7f0f714bf700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f6c10c530 0x7f0f6c19c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:38.396 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.414+0000 7f0f714bf700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6c19ce90 con 0x7f0f6c10c530 2023-07-14T12:00:38.396 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.414+0000 7f0f714bf700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6c19d000 con 0x7f0f6c0fe980 2023-07-14T12:00:38.396 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.414+0000 7f0f714bf700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6c19d1c0 con 0x7f0f6c0ff290 2023-07-14T12:00:38.397 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.414+0000 7f0f6b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f6c10c530 0x7f0f6c19c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:38.397 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.414+0000 7f0f6b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f6c10c530 0x7f0f6c19c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50814/0 (socket says 172.21.15.17:50814) 2023-07-14T12:00:38.397 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.414+0000 7f0f6affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f6c0fe980 0x7f0f6c197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:38.397 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.414+0000 7f0f6b7fe700 1 -- 172.21.15.17:0/1990556335 learned_addr learned my addr 172.21.15.17:0/1990556335 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:00:38.397 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.414+0000 7f0f6a7fc700 1 --2- 172.21.15.17:0/1990556335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f6c0ff290 0x7f0f6c198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:38.398 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.414+0000 7f0f6b7fe700 1 -- 172.21.15.17:0/1990556335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f6c0ff290 msgr2=0x7f0f6c198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:38.398 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.414+0000 7f0f6b7fe700 1 --2- 172.21.15.17:0/1990556335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f6c0ff290 0x7f0f6c198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:38.398 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.414+0000 7f0f6b7fe700 1 -- 172.21.15.17:0/1990556335 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f6c0fe980 msgr2=0x7f0f6c197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:38.398 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.414+0000 7f0f6b7fe700 1 --2- 172.21.15.17:0/1990556335 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f6c0fe980 0x7f0f6c197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:38.398 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.414+0000 7f0f6b7fe700 1 -- 172.21.15.17:0/1990556335 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f600095f0 con 0x7f0f6c10c530 2023-07-14T12:00:38.398 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.415+0000 7f0f6affd700 1 --2- 172.21.15.17:0/1990556335 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f6c0fe980 0x7f0f6c197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T12:00:38.398 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.415+0000 7f0f6b7fe700 1 --2- 172.21.15.17:0/1990556335 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f6c10c530 0x7f0f6c19c880 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0f6400ef90 tx=0x7f0f6400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:38.398 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.415+0000 7f0f53fff700 1 -- 172.21.15.17:0/1990556335 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f64018070 con 0x7f0f6c10c530 2023-07-14T12:00:38.399 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.415+0000 7f0f714bf700 1 -- 172.21.15.17:0/1990556335 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f6c19d440 con 0x7f0f6c10c530 2023-07-14T12:00:38.399 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.415+0000 7f0f53fff700 1 -- 172.21.15.17:0/1990556335 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f6400f040 con 0x7f0f6c10c530 2023-07-14T12:00:38.401 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.415+0000 7f0f53fff700 1 -- 172.21.15.17:0/1990556335 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6400ccc0 con 0x7f0f6c10c530 2023-07-14T12:00:38.401 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.415+0000 7f0f714bf700 1 -- 172.21.15.17:0/1990556335 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f6c1a4d00 con 0x7f0f6c10c530 2023-07-14T12:00:38.402 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.416+0000 7f0f714bf700 1 -- 172.21.15.17:0/1990556335 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0f6c04e210 con 0x7f0f6c10c530 2023-07-14T12:00:38.402 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.418+0000 7f0f53fff700 1 -- 172.21.15.17:0/1990556335 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f0f64003a00 con 0x7f0f6c10c530 2023-07-14T12:00:38.402 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.420+0000 7f0f53fff700 1 --2- 172.21.15.17:0/1990556335 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f0f540a1150 0x7f0f540a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:38.402 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.420+0000 7f0f53fff700 1 -- 172.21.15.17:0/1990556335 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0f640ccf80 con 0x7f0f6c10c530 2023-07-14T12:00:38.402 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.420+0000 7f0f6affd700 1 --2- 172.21.15.17:0/1990556335 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f0f540a1150 0x7f0f540a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:38.402 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.420+0000 7f0f53fff700 1 --2- 172.21.15.17:0/1990556335 >> [v2:172.21.15.17:6810/2784619821,v1:172.21.15.17:6811/2784619821] conn(0x7f0f540ac160 0x7f0f540ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:38.403 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.420+0000 7f0f53fff700 1 -- 172.21.15.17:0/1990556335 --> [v2:172.21.15.17:6810/2784619821,v1:172.21.15.17:6811/2784619821] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0f540aecc0 con 0x7f0f540ac160 2023-07-14T12:00:38.403 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.420+0000 7f0f53fff700 1 -- 172.21.15.17:0/1990556335 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f640cd2f0 con 0x7f0f6c10c530 2023-07-14T12:00:38.403 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.421+0000 7f0f6a7fc700 1 --2- 172.21.15.17:0/1990556335 >> [v2:172.21.15.17:6810/2784619821,v1:172.21.15.17:6811/2784619821] conn(0x7f0f540ac160 0x7f0f540ae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:38.404 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.421+0000 7f0f6affd700 1 --2- 172.21.15.17:0/1990556335 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f0f540a1150 0x7f0f540a3600 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c011490 tx=0x7f0f5c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:38.404 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.421+0000 7f0f6a7fc700 1 --2- 172.21.15.17:0/1990556335 >> [v2:172.21.15.17:6810/2784619821,v1:172.21.15.17:6811/2784619821] conn(0x7f0f540ac160 0x7f0f540ae5f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:38.404 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.422+0000 7f0f53fff700 1 -- 172.21.15.17:0/1990556335 <== osd.3 v2:172.21.15.17:6810/2784619821 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f0f540aecc0 con 0x7f0f540ac160 2023-07-14T12:00:38.419 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.439+0000 7f0f714bf700 1 -- 172.21.15.17:0/1990556335 --> [v2:172.21.15.17:6810/2784619821,v1:172.21.15.17:6811/2784619821] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0f6c111fd0 con 0x7f0f540ac160 2023-07-14T12:00:38.420 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.439+0000 7f0f53fff700 1 -- 172.21.15.17:0/1990556335 <== osd.3 v2:172.21.15.17:6810/2784619821 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f0f6001e040 con 0x7f0f540ac160 2023-07-14T12:00:38.420 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.440+0000 7f0f714bf700 1 -- 172.21.15.17:0/1990556335 >> [v2:172.21.15.17:6810/2784619821,v1:172.21.15.17:6811/2784619821] conn(0x7f0f540ac160 msgr2=0x7f0f540ae5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:38.421 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.440+0000 7f0f714bf700 1 --2- 172.21.15.17:0/1990556335 >> [v2:172.21.15.17:6810/2784619821,v1:172.21.15.17:6811/2784619821] conn(0x7f0f540ac160 0x7f0f540ae5f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:38.421 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.440+0000 7f0f714bf700 1 -- 172.21.15.17:0/1990556335 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f0f540a1150 msgr2=0x7f0f540a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:38.421 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.440+0000 7f0f714bf700 1 --2- 172.21.15.17:0/1990556335 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f0f540a1150 0x7f0f540a3600 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c011490 tx=0x7f0f5c009260 comp rx=0 tx=0).stop 2023-07-14T12:00:38.421 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.440+0000 7f0f714bf700 1 -- 172.21.15.17:0/1990556335 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f6c10c530 msgr2=0x7f0f6c19c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:38.421 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.440+0000 7f0f714bf700 1 --2- 172.21.15.17:0/1990556335 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f6c10c530 0x7f0f6c19c880 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0f6400ef90 tx=0x7f0f6400c3a0 comp rx=0 tx=0).stop 2023-07-14T12:00:38.421 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.440+0000 7f0f6b7fe700 1 -- 172.21.15.17:0/1990556335 reap_dead start 2023-07-14T12:00:38.422 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.440+0000 7f0f714bf700 1 -- 172.21.15.17:0/1990556335 shutdown_connections 2023-07-14T12:00:38.422 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.440+0000 7f0f714bf700 1 -- 172.21.15.17:0/1990556335 >> 172.21.15.17:0/1990556335 conn(0x7f0f6c075b70 msgr2=0x7f0f6c0fc3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:38.422 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.440+0000 7f0f714bf700 1 -- 172.21.15.17:0/1990556335 shutdown_connections 2023-07-14T12:00:38.422 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.441+0000 7f0f714bf700 1 -- 172.21.15.17:0/1990556335 wait complete. 2023-07-14T12:00:38.967 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.687+0000 7fd13b3dd700 1 -- 172.21.15.17:0/821990928 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1341007a0 msgr2=0x7fd134104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:38.968 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.687+0000 7fd13b3dd700 1 --2- 172.21.15.17:0/821990928 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1341007a0 0x7fd134104770 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd1240099c0 tx=0x7fd124009cd0 comp rx=0 tx=0).stop 2023-07-14T12:00:38.968 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.687+0000 7fd13b3dd700 1 -- 172.21.15.17:0/821990928 shutdown_connections 2023-07-14T12:00:38.968 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.687+0000 7fd13b3dd700 1 --2- 172.21.15.17:0/821990928 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd134104ea0 0x7fd13410ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:38.969 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.687+0000 7fd13b3dd700 1 --2- 172.21.15.17:0/821990928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1340ffe00 0x7fd1341001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:38.969 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.687+0000 7fd13b3dd700 1 --2- 172.21.15.17:0/821990928 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1341007a0 0x7fd134104770 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:38.969 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.687+0000 7fd13b3dd700 1 -- 172.21.15.17:0/821990928 >> 172.21.15.17:0/821990928 conn(0x7fd1340fb660 msgr2=0x7fd1340fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:38.969 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.687+0000 7fd13b3dd700 1 -- 172.21.15.17:0/821990928 shutdown_connections 2023-07-14T12:00:38.969 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.687+0000 7fd13b3dd700 1 -- 172.21.15.17:0/821990928 wait complete. 2023-07-14T12:00:38.970 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.688+0000 7fd13b3dd700 1 Processor -- start 2023-07-14T12:00:38.970 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.689+0000 7fd13b3dd700 1 -- start start 2023-07-14T12:00:38.970 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.689+0000 7fd13b3dd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1340ffe00 0x7fd134197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:38.970 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.689+0000 7fd13b3dd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1341007a0 0x7fd134198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:38.970 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.689+0000 7fd13b3dd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd134104ea0 0x7fd13419c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:38.970 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.689+0000 7fd13b3dd700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13419ced0 con 0x7fd1340ffe00 2023-07-14T12:00:38.970 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.689+0000 7fd13b3dd700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13419d040 con 0x7fd1341007a0 2023-07-14T12:00:38.971 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.689+0000 7fd13b3dd700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13419d200 con 0x7fd134104ea0 2023-07-14T12:00:38.971 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.689+0000 7fd139179700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1340ffe00 0x7fd134197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:38.971 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.689+0000 7fd139179700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1340ffe00 0x7fd134197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50800/0 (socket says 172.21.15.17:50800) 2023-07-14T12:00:38.971 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.689+0000 7fd139179700 1 -- 172.21.15.17:0/937980357 learned_addr learned my addr 172.21.15.17:0/937980357 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:00:38.971 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.689+0000 7fd138978700 1 --2- 172.21.15.17:0/937980357 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1341007a0 0x7fd134198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:38.971 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.690+0000 7fd13997a700 1 --2- 172.21.15.17:0/937980357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd134104ea0 0x7fd13419c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:38.971 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.690+0000 7fd139179700 1 -- 172.21.15.17:0/937980357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd134104ea0 msgr2=0x7fd13419c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:38.971 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.690+0000 7fd139179700 1 --2- 172.21.15.17:0/937980357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd134104ea0 0x7fd13419c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:38.972 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.690+0000 7fd139179700 1 -- 172.21.15.17:0/937980357 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1341007a0 msgr2=0x7fd134198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:38.972 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.690+0000 7fd139179700 1 --2- 172.21.15.17:0/937980357 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1341007a0 0x7fd134198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:38.972 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.690+0000 7fd139179700 1 -- 172.21.15.17:0/937980357 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1240095f0 con 0x7fd1340ffe00 2023-07-14T12:00:38.972 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.690+0000 7fd138978700 1 --2- 172.21.15.17:0/937980357 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1341007a0 0x7fd134198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T12:00:38.972 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.690+0000 7fd139179700 1 --2- 172.21.15.17:0/937980357 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1340ffe00 0x7fd134197f10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd13000e9a0 tx=0x7fd13000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:38.972 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.690+0000 7fd12a7fc700 1 -- 172.21.15.17:0/937980357 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd130009790 con 0x7fd1340ffe00 2023-07-14T12:00:38.972 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.690+0000 7fd13b3dd700 1 -- 172.21.15.17:0/937980357 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd13419d4e0 con 0x7fd1340ffe00 2023-07-14T12:00:38.973 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.690+0000 7fd12a7fc700 1 -- 172.21.15.17:0/937980357 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd13000cbd0 con 0x7fd1340ffe00 2023-07-14T12:00:38.973 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.690+0000 7fd12a7fc700 1 -- 172.21.15.17:0/937980357 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd130018680 con 0x7fd1340ffe00 2023-07-14T12:00:38.973 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.690+0000 7fd13b3dd700 1 -- 172.21.15.17:0/937980357 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1341a4d40 con 0x7fd1340ffe00 2023-07-14T12:00:38.973 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.691+0000 7fd13b3dd700 1 -- 172.21.15.17:0/937980357 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd13404e210 con 0x7fd1340ffe00 2023-07-14T12:00:38.973 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.693+0000 7fd12a7fc700 1 -- 172.21.15.17:0/937980357 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd13000cd40 con 0x7fd1340ffe00 2023-07-14T12:00:38.973 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.695+0000 7fd12a7fc700 1 --2- 172.21.15.17:0/937980357 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fd1200a1150 0x7fd1200a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:38.973 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.695+0000 7fd12a7fc700 1 -- 172.21.15.17:0/937980357 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd130014070 con 0x7fd1340ffe00 2023-07-14T12:00:38.973 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.695+0000 7fd138978700 1 --2- 172.21.15.17:0/937980357 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fd1200a1150 0x7fd1200a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:38.974 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.695+0000 7fd12a7fc700 1 --2- 172.21.15.17:0/937980357 >> [v2:172.21.15.17:6802/3588366384,v1:172.21.15.17:6803/3588366384] conn(0x7fd1200ac160 0x7fd1200ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:38.974 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.695+0000 7fd12a7fc700 1 -- 172.21.15.17:0/937980357 --> [v2:172.21.15.17:6802/3588366384,v1:172.21.15.17:6803/3588366384] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd1200aecc0 con 0x7fd1200ac160 2023-07-14T12:00:38.974 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.695+0000 7fd12a7fc700 1 -- 172.21.15.17:0/937980357 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1300cd990 con 0x7fd1340ffe00 2023-07-14T12:00:38.974 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.695+0000 7fd13997a700 1 --2- 172.21.15.17:0/937980357 >> [v2:172.21.15.17:6802/3588366384,v1:172.21.15.17:6803/3588366384] conn(0x7fd1200ac160 0x7fd1200ae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:38.974 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.696+0000 7fd138978700 1 --2- 172.21.15.17:0/937980357 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fd1200a1150 0x7fd1200a3600 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd1240098e0 tx=0x7fd12401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:38.974 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.696+0000 7fd13997a700 1 --2- 172.21.15.17:0/937980357 >> [v2:172.21.15.17:6802/3588366384,v1:172.21.15.17:6803/3588366384] conn(0x7fd1200ac160 0x7fd1200ae5f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:38.975 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.697+0000 7fd12a7fc700 1 -- 172.21.15.17:0/937980357 <== osd.0 v2:172.21.15.17:6802/3588366384 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fd1200aecc0 con 0x7fd1200ac160 2023-07-14T12:00:38.975 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.713+0000 7fd13b3dd700 1 -- 172.21.15.17:0/937980357 --> [v2:172.21.15.17:6802/3588366384,v1:172.21.15.17:6803/3588366384] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd13410fd90 con 0x7fd1200ac160 2023-07-14T12:00:38.975 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.714+0000 7fd12a7fc700 1 -- 172.21.15.17:0/937980357 <== osd.0 v2:172.21.15.17:6802/3588366384 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd12c00f040 con 0x7fd1200ac160 2023-07-14T12:00:38.975 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.714+0000 7fd13b3dd700 1 -- 172.21.15.17:0/937980357 >> [v2:172.21.15.17:6802/3588366384,v1:172.21.15.17:6803/3588366384] conn(0x7fd1200ac160 msgr2=0x7fd1200ae5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:38.976 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.714+0000 7fd13b3dd700 1 --2- 172.21.15.17:0/937980357 >> [v2:172.21.15.17:6802/3588366384,v1:172.21.15.17:6803/3588366384] conn(0x7fd1200ac160 0x7fd1200ae5f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:38.976 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.714+0000 7fd13b3dd700 1 -- 172.21.15.17:0/937980357 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fd1200a1150 msgr2=0x7fd1200a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:38.976 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.714+0000 7fd13b3dd700 1 --2- 172.21.15.17:0/937980357 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fd1200a1150 0x7fd1200a3600 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd1240098e0 tx=0x7fd12401a040 comp rx=0 tx=0).stop 2023-07-14T12:00:38.976 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.714+0000 7fd13b3dd700 1 -- 172.21.15.17:0/937980357 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1340ffe00 msgr2=0x7fd134197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:38.976 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.714+0000 7fd13b3dd700 1 --2- 172.21.15.17:0/937980357 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1340ffe00 0x7fd134197f10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd13000e9a0 tx=0x7fd13000ed60 comp rx=0 tx=0).stop 2023-07-14T12:00:38.976 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.714+0000 7fd13997a700 1 -- 172.21.15.17:0/937980357 reap_dead start 2023-07-14T12:00:38.976 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.714+0000 7fd13b3dd700 1 -- 172.21.15.17:0/937980357 shutdown_connections 2023-07-14T12:00:38.977 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.714+0000 7fd13b3dd700 1 -- 172.21.15.17:0/937980357 >> 172.21.15.17:0/937980357 conn(0x7fd1340fb660 msgr2=0x7fd1340fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:38.977 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.715+0000 7fd13b3dd700 1 -- 172.21.15.17:0/937980357 shutdown_connections 2023-07-14T12:00:38.977 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:37.715+0000 7fd13b3dd700 1 -- 172.21.15.17:0/937980357 wait complete. 2023-07-14T12:00:38.993 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.474+0000 7f7d84f70700 1 -- 172.21.15.17:0/3757133609 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7d80101700 msgr2=0x7f7d80109c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:38.993 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.474+0000 7f7d84f70700 1 --2- 172.21.15.17:0/3757133609 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7d80101700 0x7f7d80109c00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7d74009990 tx=0x7f7d74009ca0 comp rx=0 tx=0).stop 2023-07-14T12:00:38.993 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.474+0000 7f7d84f70700 1 -- 172.21.15.17:0/3757133609 shutdown_connections 2023-07-14T12:00:38.994 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.474+0000 7f7d84f70700 1 --2- 172.21.15.17:0/3757133609 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d8010a140 0x7f7d8010c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:38.994 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.474+0000 7f7d84f70700 1 --2- 172.21.15.17:0/3757133609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d80100df0 0x7f7d801011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:38.994 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.474+0000 7f7d84f70700 1 --2- 172.21.15.17:0/3757133609 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7d80101700 0x7f7d80109c00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:38.995 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.474+0000 7f7d84f70700 1 -- 172.21.15.17:0/3757133609 >> 172.21.15.17:0/3757133609 conn(0x7f7d80075b70 msgr2=0x7f7d80075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:38.995 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.474+0000 7f7d84f70700 1 -- 172.21.15.17:0/3757133609 shutdown_connections 2023-07-14T12:00:38.995 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.474+0000 7f7d84f70700 1 -- 172.21.15.17:0/3757133609 wait complete. 2023-07-14T12:00:38.995 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.476+0000 7f7d84f70700 1 Processor -- start 2023-07-14T12:00:38.995 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.476+0000 7f7d84f70700 1 -- start start 2023-07-14T12:00:38.995 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.477+0000 7f7d84f70700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d80100df0 0x7f7d8019ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:38.995 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.477+0000 7f7d84f70700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d80101700 0x7f7d8019bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:38.996 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.477+0000 7f7d84f70700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7d8010a140 0x7f7d80195aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:38.996 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.477+0000 7f7d84f70700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d80196110 con 0x7f7d80100df0 2023-07-14T12:00:38.996 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.477+0000 7f7d84f70700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d80196280 con 0x7f7d8010a140 2023-07-14T12:00:38.996 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.477+0000 7f7d84f70700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d80196440 con 0x7f7d80101700 2023-07-14T12:00:38.996 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.477+0000 7f7d7e59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d80100df0 0x7f7d8019ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:38.996 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.477+0000 7f7d7e59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d80100df0 0x7f7d8019ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50822/0 (socket says 172.21.15.17:50822) 2023-07-14T12:00:38.996 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.477+0000 7f7d7dd9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d80101700 0x7f7d8019bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:38.997 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.477+0000 7f7d7e59c700 1 -- 172.21.15.17:0/3965488430 learned_addr learned my addr 172.21.15.17:0/3965488430 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:00:38.997 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.477+0000 7f7d7ed9d700 1 --2- 172.21.15.17:0/3965488430 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7d8010a140 0x7f7d80195aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:38.997 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.478+0000 7f7d7dd9b700 1 -- 172.21.15.17:0/3965488430 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7d8010a140 msgr2=0x7f7d80195aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:38.997 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.478+0000 7f7d7dd9b700 1 --2- 172.21.15.17:0/3965488430 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7d8010a140 0x7f7d80195aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:38.997 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.478+0000 7f7d7dd9b700 1 -- 172.21.15.17:0/3965488430 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d80100df0 msgr2=0x7f7d8019ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:38.997 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.478+0000 7f7d7dd9b700 1 --2- 172.21.15.17:0/3965488430 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d80100df0 0x7f7d8019ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:38.997 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.478+0000 7f7d7dd9b700 1 -- 172.21.15.17:0/3965488430 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d740095f0 con 0x7f7d80101700 2023-07-14T12:00:38.997 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.478+0000 7f7d7dd9b700 1 --2- 172.21.15.17:0/3965488430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d80101700 0x7f7d8019bf60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7d74009990 tx=0x7f7d74009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:38.998 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.478+0000 7f7d6f7fe700 1 -- 172.21.15.17:0/3965488430 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d7401d070 con 0x7f7d80101700 2023-07-14T12:00:38.998 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.478+0000 7f7d6f7fe700 1 -- 172.21.15.17:0/3965488430 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d74021410 con 0x7f7d80101700 2023-07-14T12:00:38.998 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.478+0000 7f7d84f70700 1 -- 172.21.15.17:0/3965488430 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d801966c0 con 0x7f7d80101700 2023-07-14T12:00:38.998 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.478+0000 7f7d6f7fe700 1 -- 172.21.15.17:0/3965488430 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d74017650 con 0x7f7d80101700 2023-07-14T12:00:38.998 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.478+0000 7f7d84f70700 1 -- 172.21.15.17:0/3965488430 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d801a2a10 con 0x7f7d80101700 2023-07-14T12:00:38.998 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.479+0000 7f7d84f70700 1 -- 172.21.15.17:0/3965488430 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7d8004e210 con 0x7f7d80101700 2023-07-14T12:00:38.999 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.481+0000 7f7d6f7fe700 1 -- 172.21.15.17:0/3965488430 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7d74021810 con 0x7f7d80101700 2023-07-14T12:00:38.999 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.483+0000 7f7d6f7fe700 1 --2- 172.21.15.17:0/3965488430 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f7d680a1150 0x7f7d680a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:38.999 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.483+0000 7f7d6f7fe700 1 -- 172.21.15.17:0/3965488430 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7d740cdf20 con 0x7f7d80101700 2023-07-14T12:00:38.999 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.483+0000 7f7d7e59c700 1 --2- 172.21.15.17:0/3965488430 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f7d680a1150 0x7f7d680a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:39.000 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.483+0000 7f7d6f7fe700 1 --2- 172.21.15.17:0/3965488430 >> [v2:172.21.15.97:6800/1877269815,v1:172.21.15.97:6801/1877269815] conn(0x7f7d680ac160 0x7f7d680ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:39.000 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.483+0000 7f7d6f7fe700 1 -- 172.21.15.17:0/3965488430 --> [v2:172.21.15.97:6800/1877269815,v1:172.21.15.97:6801/1877269815] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7d680aecc0 con 0x7f7d680ac160 2023-07-14T12:00:39.000 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.483+0000 7f7d6f7fe700 1 -- 172.21.15.17:0/3965488430 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d740d0500 con 0x7f7d80101700 2023-07-14T12:00:39.001 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.484+0000 7f7d7ed9d700 1 --2- 172.21.15.17:0/3965488430 >> [v2:172.21.15.97:6800/1877269815,v1:172.21.15.97:6801/1877269815] conn(0x7f7d680ac160 0x7f7d680ae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:39.001 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.484+0000 7f7d7e59c700 1 --2- 172.21.15.17:0/3965488430 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f7d680a1150 0x7f7d680a3600 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7d70009d20 tx=0x7f7d70009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:39.001 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.484+0000 7f7d7ed9d700 1 --2- 172.21.15.17:0/3965488430 >> [v2:172.21.15.97:6800/1877269815,v1:172.21.15.97:6801/1877269815] conn(0x7f7d680ac160 0x7f7d680ae5f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:39.001 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.485+0000 7f7d6f7fe700 1 -- 172.21.15.17:0/3965488430 <== osd.2 v2:172.21.15.97:6800/1877269815 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f7d680aecc0 con 0x7f7d680ac160 2023-07-14T12:00:39.001 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.503+0000 7f7d84f70700 1 -- 172.21.15.17:0/3965488430 --> [v2:172.21.15.97:6800/1877269815,v1:172.21.15.97:6801/1877269815] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7d800fdca0 con 0x7f7d680ac160 2023-07-14T12:00:39.001 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.504+0000 7f7d6f7fe700 1 -- 172.21.15.17:0/3965488430 <== osd.2 v2:172.21.15.97:6800/1877269815 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7d800fdca0 con 0x7f7d680ac160 2023-07-14T12:00:39.001 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.504+0000 7f7d84f70700 1 -- 172.21.15.17:0/3965488430 >> [v2:172.21.15.97:6800/1877269815,v1:172.21.15.97:6801/1877269815] conn(0x7f7d680ac160 msgr2=0x7f7d680ae5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:39.002 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.504+0000 7f7d84f70700 1 --2- 172.21.15.17:0/3965488430 >> [v2:172.21.15.97:6800/1877269815,v1:172.21.15.97:6801/1877269815] conn(0x7f7d680ac160 0x7f7d680ae5f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:39.002 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.504+0000 7f7d84f70700 1 -- 172.21.15.17:0/3965488430 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f7d680a1150 msgr2=0x7f7d680a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:39.002 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.504+0000 7f7d84f70700 1 --2- 172.21.15.17:0/3965488430 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f7d680a1150 0x7f7d680a3600 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7d70009d20 tx=0x7f7d70009260 comp rx=0 tx=0).stop 2023-07-14T12:00:39.002 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.504+0000 7f7d84f70700 1 -- 172.21.15.17:0/3965488430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d80101700 msgr2=0x7f7d8019bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:39.002 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.504+0000 7f7d84f70700 1 --2- 172.21.15.17:0/3965488430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d80101700 0x7f7d8019bf60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7d74009990 tx=0x7f7d74009580 comp rx=0 tx=0).stop 2023-07-14T12:00:39.002 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.504+0000 7f7d7ed9d700 1 -- 172.21.15.17:0/3965488430 reap_dead start 2023-07-14T12:00:39.002 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.504+0000 7f7d84f70700 1 -- 172.21.15.17:0/3965488430 shutdown_connections 2023-07-14T12:00:39.003 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.504+0000 7f7d84f70700 1 -- 172.21.15.17:0/3965488430 >> 172.21.15.17:0/3965488430 conn(0x7f7d80075b70 msgr2=0x7f7d800fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:39.003 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.504+0000 7f7d84f70700 1 -- 172.21.15.17:0/3965488430 shutdown_connections 2023-07-14T12:00:39.003 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:38.504+0000 7f7d84f70700 1 -- 172.21.15.17:0/3965488430 wait complete. 2023-07-14T12:00:39.084 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:38 smithi017 ceph-mon[126339]: pgmap v306: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-07-14T12:00:39.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:38 smithi093 ceph-mon[128922]: pgmap v306: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-07-14T12:00:39.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:38 smithi097 ceph-mon[129313]: pgmap v306: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-07-14T12:00:40.191 INFO:teuthology.orchestra.run.smithi017.stdout:77309411393 2023-07-14T12:00:40.191 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.2 2023-07-14T12:00:40.193 INFO:teuthology.orchestra.run.smithi017.stdout:34359738440 2023-07-14T12:00:40.193 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.0 2023-07-14T12:00:41.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:40 smithi093 ceph-mon[128922]: pgmap v307: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-07-14T12:00:41.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:40 smithi097 ceph-mon[129313]: pgmap v307: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-07-14T12:00:41.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:40 smithi017 ceph-mon[126339]: pgmap v307: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-07-14T12:00:41.664 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.191+0000 7fbb9ee3e700 1 -- 172.21.15.17:0/3993911909 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb9806da70 msgr2=0x7fbb98107510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:41.664 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.191+0000 7fbb9ee3e700 1 --2- 172.21.15.17:0/3993911909 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb9806da70 0x7fbb98107510 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c009990 tx=0x7fbb8c009ca0 comp rx=0 tx=0).stop 2023-07-14T12:00:41.665 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.191+0000 7fbb9ee3e700 1 -- 172.21.15.17:0/3993911909 shutdown_connections 2023-07-14T12:00:41.665 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.191+0000 7fbb9ee3e700 1 --2- 172.21.15.17:0/3993911909 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb98112930 0x7fbb98114d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:41.665 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.191+0000 7fbb9ee3e700 1 --2- 172.21.15.17:0/3993911909 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb9806da70 0x7fbb98107510 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:41.665 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.191+0000 7fbb9ee3e700 1 --2- 172.21.15.17:0/3993911909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb98107ae0 0x7fbb98107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:41.665 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.191+0000 7fbb9ee3e700 1 -- 172.21.15.17:0/3993911909 >> 172.21.15.17:0/3993911909 conn(0x7fbb9806be40 msgr2=0x7fbb98071020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:41.665 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.191+0000 7fbb9ee3e700 1 -- 172.21.15.17:0/3993911909 shutdown_connections 2023-07-14T12:00:41.665 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.191+0000 7fbb9ee3e700 1 -- 172.21.15.17:0/3993911909 wait complete. 2023-07-14T12:00:41.666 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.193+0000 7fbb9ee3e700 1 Processor -- start 2023-07-14T12:00:41.666 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.193+0000 7fbb9ee3e700 1 -- start start 2023-07-14T12:00:41.666 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.193+0000 7fbb9ee3e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb9806da70 0x7fbb981120a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:41.666 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.193+0000 7fbb9ee3e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb98107ae0 0x7fbb9810f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:41.666 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.193+0000 7fbb9ee3e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb98112930 0x7fbb9810f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:41.666 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.193+0000 7fbb9ee3e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9810fc50 con 0x7fbb98112930 2023-07-14T12:00:41.667 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.193+0000 7fbb9ee3e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9810fdc0 con 0x7fbb9806da70 2023-07-14T12:00:41.667 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.193+0000 7fbb9ee3e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb981b1100 con 0x7fbb98107ae0 2023-07-14T12:00:41.667 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.194+0000 7fbb9cbda700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb9806da70 0x7fbb981120a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:41.667 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.194+0000 7fbb9cbda700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb9806da70 0x7fbb981120a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.17:46600/0 (socket says 172.21.15.17:46600) 2023-07-14T12:00:41.667 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.194+0000 7fbb9cbda700 1 -- 172.21.15.17:0/611803700 learned_addr learned my addr 172.21.15.17:0/611803700 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:00:41.667 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.194+0000 7fbb97fff700 1 --2- 172.21.15.17:0/611803700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb98107ae0 0x7fbb9810f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:41.667 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.194+0000 7fbb9d3db700 1 --2- 172.21.15.17:0/611803700 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb98112930 0x7fbb9810f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:41.668 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.194+0000 7fbb9cbda700 1 -- 172.21.15.17:0/611803700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb98107ae0 msgr2=0x7fbb9810f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:41.668 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.194+0000 7fbb9cbda700 1 --2- 172.21.15.17:0/611803700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb98107ae0 0x7fbb9810f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:41.668 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.194+0000 7fbb9cbda700 1 -- 172.21.15.17:0/611803700 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb98112930 msgr2=0x7fbb9810f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:41.668 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.194+0000 7fbb9cbda700 1 --2- 172.21.15.17:0/611803700 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb98112930 0x7fbb9810f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:41.668 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.194+0000 7fbb9cbda700 1 -- 172.21.15.17:0/611803700 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb8c0095f0 con 0x7fbb9806da70 2023-07-14T12:00:41.668 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.194+0000 7fbb9d3db700 1 --2- 172.21.15.17:0/611803700 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb98112930 0x7fbb9810f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T12:00:41.668 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.194+0000 7fbb9cbda700 1 --2- 172.21.15.17:0/611803700 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb9806da70 0x7fbb981120a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c000c00 tx=0x7fbb8c005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:41.669 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.195+0000 7fbb95ffb700 1 -- 172.21.15.17:0/611803700 <== mon.2 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c01d070 con 0x7fbb9806da70 2023-07-14T12:00:41.669 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.195+0000 7fbb9ee3e700 1 -- 172.21.15.17:0/611803700 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb981b12a0 con 0x7fbb9806da70 2023-07-14T12:00:41.669 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.195+0000 7fbb95ffb700 1 -- 172.21.15.17:0/611803700 <== mon.2 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb8c022470 con 0x7fbb9806da70 2023-07-14T12:00:41.669 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.195+0000 7fbb95ffb700 1 -- 172.21.15.17:0/611803700 <== mon.2 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c017650 con 0x7fbb9806da70 2023-07-14T12:00:41.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.195+0000 7fbb9ee3e700 1 -- 172.21.15.17:0/611803700 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb981b16b0 con 0x7fbb9806da70 2023-07-14T12:00:41.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.196+0000 7fbb9ee3e700 1 -- 172.21.15.17:0/611803700 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbb9804e210 con 0x7fbb9806da70 2023-07-14T12:00:41.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.198+0000 7fbb95ffb700 1 -- 172.21.15.17:0/611803700 <== mon.2 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fbb8c022850 con 0x7fbb9806da70 2023-07-14T12:00:41.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.199+0000 7fbb95ffb700 1 --2- 172.21.15.17:0/611803700 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fbb880a1150 0x7fbb880a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:41.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.199+0000 7fbb95ffb700 1 -- 172.21.15.17:0/611803700 <== mon.2 v2:172.21.15.93:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fbb8c0ce8f0 con 0x7fbb9806da70 2023-07-14T12:00:41.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.200+0000 7fbb97fff700 1 --2- 172.21.15.17:0/611803700 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fbb880a1150 0x7fbb880a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:41.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.200+0000 7fbb95ffb700 1 --2- 172.21.15.17:0/611803700 >> [v2:172.21.15.97:6808/2184413924,v1:172.21.15.97:6809/2184413924] conn(0x7fbb880ac160 0x7fbb880ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:41.671 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.200+0000 7fbb95ffb700 1 -- 172.21.15.17:0/611803700 --> [v2:172.21.15.97:6808/2184413924,v1:172.21.15.97:6809/2184413924] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbb880aecc0 con 0x7fbb880ac160 2023-07-14T12:00:41.671 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.200+0000 7fbb95ffb700 1 -- 172.21.15.17:0/611803700 <== mon.2 v2:172.21.15.93:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb8c0cec60 con 0x7fbb9806da70 2023-07-14T12:00:41.671 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.200+0000 7fbb9d3db700 1 --2- 172.21.15.17:0/611803700 >> [v2:172.21.15.97:6808/2184413924,v1:172.21.15.97:6809/2184413924] conn(0x7fbb880ac160 0x7fbb880ae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:41.671 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.200+0000 7fbb97fff700 1 --2- 172.21.15.17:0/611803700 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fbb880a1150 0x7fbb880a3600 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbb98110910 tx=0x7fbb84009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:41.671 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.201+0000 7fbb9d3db700 1 --2- 172.21.15.17:0/611803700 >> [v2:172.21.15.97:6808/2184413924,v1:172.21.15.97:6809/2184413924] conn(0x7fbb880ac160 0x7fbb880ae5f0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:41.671 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.201+0000 7fbb95ffb700 1 -- 172.21.15.17:0/611803700 <== osd.5 v2:172.21.15.97:6808/2184413924 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fbb880aecc0 con 0x7fbb880ac160 2023-07-14T12:00:41.671 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.219+0000 7fbb9ee3e700 1 -- 172.21.15.17:0/611803700 --> [v2:172.21.15.97:6808/2184413924,v1:172.21.15.97:6809/2184413924] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbb9810a420 con 0x7fbb880ac160 2023-07-14T12:00:41.672 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.220+0000 7fbb95ffb700 1 -- 172.21.15.17:0/611803700 <== osd.5 v2:172.21.15.97:6808/2184413924 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fbb90011040 con 0x7fbb880ac160 2023-07-14T12:00:41.672 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.221+0000 7fbb9ee3e700 1 -- 172.21.15.17:0/611803700 >> [v2:172.21.15.97:6808/2184413924,v1:172.21.15.97:6809/2184413924] conn(0x7fbb880ac160 msgr2=0x7fbb880ae5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:41.672 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.221+0000 7fbb9ee3e700 1 --2- 172.21.15.17:0/611803700 >> [v2:172.21.15.97:6808/2184413924,v1:172.21.15.97:6809/2184413924] conn(0x7fbb880ac160 0x7fbb880ae5f0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:41.672 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.221+0000 7fbb9ee3e700 1 -- 172.21.15.17:0/611803700 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fbb880a1150 msgr2=0x7fbb880a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:41.672 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.221+0000 7fbb9ee3e700 1 --2- 172.21.15.17:0/611803700 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fbb880a1150 0x7fbb880a3600 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbb98110910 tx=0x7fbb84009260 comp rx=0 tx=0).stop 2023-07-14T12:00:41.673 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.221+0000 7fbb9ee3e700 1 -- 172.21.15.17:0/611803700 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb9806da70 msgr2=0x7fbb981120a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:41.673 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.221+0000 7fbb9ee3e700 1 --2- 172.21.15.17:0/611803700 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb9806da70 0x7fbb981120a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c000c00 tx=0x7fbb8c005720 comp rx=0 tx=0).stop 2023-07-14T12:00:41.673 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.221+0000 7fbb9d3db700 1 -- 172.21.15.17:0/611803700 reap_dead start 2023-07-14T12:00:41.673 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.221+0000 7fbb9ee3e700 1 -- 172.21.15.17:0/611803700 shutdown_connections 2023-07-14T12:00:41.673 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.221+0000 7fbb9ee3e700 1 -- 172.21.15.17:0/611803700 >> 172.21.15.17:0/611803700 conn(0x7fbb9806be40 msgr2=0x7fbb98116a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:41.674 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.221+0000 7fbb9ee3e700 1 -- 172.21.15.17:0/611803700 shutdown_connections 2023-07-14T12:00:41.674 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.221+0000 7fbb9ee3e700 1 -- 172.21.15.17:0/611803700 wait complete. 2023-07-14T12:00:41.695 INFO:teuthology.orchestra.run.smithi017.stdout:111669149756 2023-07-14T12:00:41.696 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.3 2023-07-14T12:00:41.876 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.157+0000 7efc803c2700 1 -- 172.21.15.17:0/2849290334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc78101700 msgr2=0x7efc78109c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:41.876 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.157+0000 7efc803c2700 1 --2- 172.21.15.17:0/2849290334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc78101700 0x7efc78109c00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efc68009990 tx=0x7efc68009ca0 comp rx=0 tx=0).stop 2023-07-14T12:00:41.876 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.157+0000 7efc803c2700 1 -- 172.21.15.17:0/2849290334 shutdown_connections 2023-07-14T12:00:41.877 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.157+0000 7efc803c2700 1 --2- 172.21.15.17:0/2849290334 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efc78100df0 0x7efc781011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:41.877 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.157+0000 7efc803c2700 1 --2- 172.21.15.17:0/2849290334 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc7810a140 0x7efc7810c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:41.877 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.157+0000 7efc803c2700 1 --2- 172.21.15.17:0/2849290334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc78101700 0x7efc78109c00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:41.877 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.157+0000 7efc803c2700 1 -- 172.21.15.17:0/2849290334 >> 172.21.15.17:0/2849290334 conn(0x7efc78075b70 msgr2=0x7efc78075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:41.877 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.157+0000 7efc803c2700 1 -- 172.21.15.17:0/2849290334 shutdown_connections 2023-07-14T12:00:41.877 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.157+0000 7efc803c2700 1 -- 172.21.15.17:0/2849290334 wait complete. 2023-07-14T12:00:41.877 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.158+0000 7efc803c2700 1 Processor -- start 2023-07-14T12:00:41.878 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.159+0000 7efc803c2700 1 -- start start 2023-07-14T12:00:41.878 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.159+0000 7efc803c2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc78100df0 0x7efc780fea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:41.878 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.159+0000 7efc803c2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efc78101700 0x7efc780fef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:41.878 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.159+0000 7efc803c2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc7810a140 0x7efc780ff510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:41.878 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.159+0000 7efc803c2700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc781a40c0 con 0x7efc78100df0 2023-07-14T12:00:41.878 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.159+0000 7efc803c2700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc781a4230 con 0x7efc78101700 2023-07-14T12:00:41.878 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.159+0000 7efc803c2700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc781a43f0 con 0x7efc7810a140 2023-07-14T12:00:41.879 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.159+0000 7efc7e15e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc78100df0 0x7efc780fea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:41.879 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.159+0000 7efc7e15e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc78100df0 0x7efc780fea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50840/0 (socket says 172.21.15.17:50840) 2023-07-14T12:00:41.879 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.160+0000 7efc7e15e700 1 -- 172.21.15.17:0/3301982237 learned_addr learned my addr 172.21.15.17:0/3301982237 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:00:41.879 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.160+0000 7efc7e95f700 1 --2- 172.21.15.17:0/3301982237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc7810a140 0x7efc780ff510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:41.879 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.160+0000 7efc7d95d700 1 --2- 172.21.15.17:0/3301982237 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efc78101700 0x7efc780fef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:41.879 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.160+0000 7efc7e15e700 1 -- 172.21.15.17:0/3301982237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc7810a140 msgr2=0x7efc780ff510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:41.879 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.160+0000 7efc7e15e700 1 --2- 172.21.15.17:0/3301982237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc7810a140 0x7efc780ff510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:41.879 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.160+0000 7efc7e15e700 1 -- 172.21.15.17:0/3301982237 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efc78101700 msgr2=0x7efc780fef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:41.880 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.160+0000 7efc7e15e700 1 --2- 172.21.15.17:0/3301982237 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efc78101700 0x7efc780fef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:41.880 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.160+0000 7efc7e15e700 1 -- 172.21.15.17:0/3301982237 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc680095f0 con 0x7efc78100df0 2023-07-14T12:00:41.880 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.160+0000 7efc7d95d700 1 --2- 172.21.15.17:0/3301982237 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efc78101700 0x7efc780fef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T12:00:41.880 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.160+0000 7efc7e95f700 1 --2- 172.21.15.17:0/3301982237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc7810a140 0x7efc780ff510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T12:00:41.880 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.160+0000 7efc7e15e700 1 --2- 172.21.15.17:0/3301982237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc78100df0 0x7efc780fea50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7efc7400ef30 tx=0x7efc7400c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:41.881 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.161+0000 7efc6f7fe700 1 -- 172.21.15.17:0/3301982237 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc74018070 con 0x7efc78100df0 2023-07-14T12:00:41.881 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.161+0000 7efc6f7fe700 1 -- 172.21.15.17:0/3301982237 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc7400f040 con 0x7efc78100df0 2023-07-14T12:00:41.881 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.161+0000 7efc803c2700 1 -- 172.21.15.17:0/3301982237 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc781a4670 con 0x7efc78100df0 2023-07-14T12:00:41.881 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.161+0000 7efc803c2700 1 -- 172.21.15.17:0/3301982237 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc781a4b70 con 0x7efc78100df0 2023-07-14T12:00:41.881 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.161+0000 7efc6f7fe700 1 -- 172.21.15.17:0/3301982237 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc74004020 con 0x7efc78100df0 2023-07-14T12:00:41.881 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.162+0000 7efc803c2700 1 -- 172.21.15.17:0/3301982237 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efc5c000ff0 con 0x7efc78100df0 2023-07-14T12:00:41.882 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.165+0000 7efc6f7fe700 1 -- 172.21.15.17:0/3301982237 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7efc7400cb20 con 0x7efc78100df0 2023-07-14T12:00:41.882 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.167+0000 7efc6f7fe700 1 --2- 172.21.15.17:0/3301982237 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7efc640a12a0 0x7efc640a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:41.882 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.167+0000 7efc6f7fe700 1 -- 172.21.15.17:0/3301982237 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7efc740cd0a0 con 0x7efc78100df0 2023-07-14T12:00:41.882 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.167+0000 7efc7d95d700 1 --2- 172.21.15.17:0/3301982237 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7efc640a12a0 0x7efc640a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:41.882 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.168+0000 7efc6f7fe700 1 --2- 172.21.15.17:0/3301982237 >> [v2:172.21.15.93:6808/3151801173,v1:172.21.15.93:6809/3151801173] conn(0x7efc640ac2b0 0x7efc640ae740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:41.882 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.168+0000 7efc6f7fe700 1 -- 172.21.15.17:0/3301982237 --> [v2:172.21.15.93:6808/3151801173,v1:172.21.15.93:6809/3151801173] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efc640aee10 con 0x7efc640ac2b0 2023-07-14T12:00:41.882 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.168+0000 7efc6f7fe700 1 -- 172.21.15.17:0/3301982237 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc740cd4b0 con 0x7efc78100df0 2023-07-14T12:00:41.882 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.168+0000 7efc7d95d700 1 --2- 172.21.15.17:0/3301982237 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7efc640a12a0 0x7efc640a3750 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efc6800b2e0 tx=0x7efc68005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:41.883 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.168+0000 7efc7e95f700 1 --2- 172.21.15.17:0/3301982237 >> [v2:172.21.15.93:6808/3151801173,v1:172.21.15.93:6809/3151801173] conn(0x7efc640ac2b0 0x7efc640ae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:41.883 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.169+0000 7efc7e95f700 1 --2- 172.21.15.17:0/3301982237 >> [v2:172.21.15.93:6808/3151801173,v1:172.21.15.93:6809/3151801173] conn(0x7efc640ac2b0 0x7efc640ae740 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:41.883 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.170+0000 7efc6f7fe700 1 -- 172.21.15.17:0/3301982237 <== osd.4 v2:172.21.15.93:6808/3151801173 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7efc640aee10 con 0x7efc640ac2b0 2023-07-14T12:00:41.883 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.188+0000 7efc803c2700 1 -- 172.21.15.17:0/3301982237 --> [v2:172.21.15.93:6808/3151801173,v1:172.21.15.93:6809/3151801173] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7efc5c002d70 con 0x7efc640ac2b0 2023-07-14T12:00:41.883 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.189+0000 7efc6f7fe700 1 -- 172.21.15.17:0/3301982237 <== osd.4 v2:172.21.15.93:6808/3151801173 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7efc5c002d70 con 0x7efc640ac2b0 2023-07-14T12:00:41.883 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.190+0000 7efc803c2700 1 -- 172.21.15.17:0/3301982237 >> [v2:172.21.15.93:6808/3151801173,v1:172.21.15.93:6809/3151801173] conn(0x7efc640ac2b0 msgr2=0x7efc640ae740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:41.883 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.190+0000 7efc803c2700 1 --2- 172.21.15.17:0/3301982237 >> [v2:172.21.15.93:6808/3151801173,v1:172.21.15.93:6809/3151801173] conn(0x7efc640ac2b0 0x7efc640ae740 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:41.883 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.190+0000 7efc803c2700 1 -- 172.21.15.17:0/3301982237 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7efc640a12a0 msgr2=0x7efc640a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:41.884 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.190+0000 7efc803c2700 1 --2- 172.21.15.17:0/3301982237 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7efc640a12a0 0x7efc640a3750 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efc6800b2e0 tx=0x7efc68005fb0 comp rx=0 tx=0).stop 2023-07-14T12:00:41.884 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.190+0000 7efc803c2700 1 -- 172.21.15.17:0/3301982237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc78100df0 msgr2=0x7efc780fea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:41.884 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.190+0000 7efc803c2700 1 --2- 172.21.15.17:0/3301982237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc78100df0 0x7efc780fea50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7efc7400ef30 tx=0x7efc7400c450 comp rx=0 tx=0).stop 2023-07-14T12:00:41.884 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.190+0000 7efc7e95f700 1 -- 172.21.15.17:0/3301982237 reap_dead start 2023-07-14T12:00:41.885 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.190+0000 7efc803c2700 1 -- 172.21.15.17:0/3301982237 shutdown_connections 2023-07-14T12:00:41.885 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.190+0000 7efc803c2700 1 -- 172.21.15.17:0/3301982237 >> 172.21.15.17:0/3301982237 conn(0x7efc78075b70 msgr2=0x7efc78101b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:41.885 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.190+0000 7efc803c2700 1 -- 172.21.15.17:0/3301982237 shutdown_connections 2023-07-14T12:00:41.885 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:41.190+0000 7efc803c2700 1 -- 172.21.15.17:0/3301982237 wait complete. 2023-07-14T12:00:43.025 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:42 smithi017 ceph-mon[126339]: pgmap v308: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-07-14T12:00:43.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:42 smithi093 ceph-mon[128922]: pgmap v308: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-07-14T12:00:43.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:42 smithi097 ceph-mon[129313]: pgmap v308: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-07-14T12:00:43.282 INFO:teuthology.orchestra.run.smithi017.stdout:167503724596 2023-07-14T12:00:43.283 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.5 2023-07-14T12:00:43.310 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.329+0000 7f526cfd9700 1 -- 172.21.15.17:0/3038872504 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f52680febe0 msgr2=0x7f52680fefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:43.311 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.329+0000 7f526cfd9700 1 --2- 172.21.15.17:0/3038872504 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f52680febe0 0x7f52680fefb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f52580098f0 tx=0x7f5258009c00 comp rx=0 tx=0).stop 2023-07-14T12:00:43.311 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.329+0000 7f526cfd9700 1 -- 172.21.15.17:0/3038872504 shutdown_connections 2023-07-14T12:00:43.311 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.329+0000 7f526cfd9700 1 --2- 172.21.15.17:0/3038872504 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f52680febe0 0x7f52680fefb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:43.311 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.329+0000 7f526cfd9700 1 --2- 172.21.15.17:0/3038872504 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5268103f10 0x7f5268104340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:43.311 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.329+0000 7f526cfd9700 1 --2- 172.21.15.17:0/3038872504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52680ff4f0 0x7f52681039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:43.311 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.329+0000 7f526cfd9700 1 -- 172.21.15.17:0/3038872504 >> 172.21.15.17:0/3038872504 conn(0x7f5268075b70 msgr2=0x7f5268075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:43.311 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.330+0000 7f526cfd9700 1 -- 172.21.15.17:0/3038872504 shutdown_connections 2023-07-14T12:00:43.312 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.330+0000 7f526cfd9700 1 -- 172.21.15.17:0/3038872504 wait complete. 2023-07-14T12:00:43.312 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.331+0000 7f526cfd9700 1 Processor -- start 2023-07-14T12:00:43.313 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.332+0000 7f526cfd9700 1 -- start start 2023-07-14T12:00:43.313 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.332+0000 7f526cfd9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52680febe0 0x7f526819c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:43.313 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.332+0000 7f526cfd9700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f52680ff4f0 0x7f526819c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:43.314 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.332+0000 7f526cfd9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5268103f10 0x7f52681a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:43.314 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.332+0000 7f526cfd9700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52681a1290 con 0x7f52680febe0 2023-07-14T12:00:43.314 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.333+0000 7f526cfd9700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52681a1400 con 0x7f52680ff4f0 2023-07-14T12:00:43.314 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.333+0000 7f526cfd9700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52681a15c0 con 0x7f5268103f10 2023-07-14T12:00:43.314 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.333+0000 7f5265d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f52680ff4f0 0x7f526819c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:43.314 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.333+0000 7f5265d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f52680ff4f0 0x7f526819c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.17:39434/0 (socket says 172.21.15.17:39434) 2023-07-14T12:00:43.314 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.333+0000 7f5265d9b700 1 -- 172.21.15.17:0/3065068049 learned_addr learned my addr 172.21.15.17:0/3065068049 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:00:43.315 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.333+0000 7f526659c700 1 --2- 172.21.15.17:0/3065068049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52680febe0 0x7f526819c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:43.315 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.334+0000 7f526659c700 1 -- 172.21.15.17:0/3065068049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5268103f10 msgr2=0x7f52681a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:43.315 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.334+0000 7f526659c700 1 --2- 172.21.15.17:0/3065068049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5268103f10 0x7f52681a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:43.316 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.334+0000 7f526659c700 1 -- 172.21.15.17:0/3065068049 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f52680ff4f0 msgr2=0x7f526819c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:43.316 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.334+0000 7f526659c700 1 --2- 172.21.15.17:0/3065068049 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f52680ff4f0 0x7f526819c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:43.316 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.334+0000 7f526659c700 1 -- 172.21.15.17:0/3065068049 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5260009d00 con 0x7f52680febe0 2023-07-14T12:00:43.316 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.334+0000 7f526659c700 1 --2- 172.21.15.17:0/3065068049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52680febe0 0x7f526819c2d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5258009810 tx=0x7f525800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:43.316 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.335+0000 7f52577fe700 1 -- 172.21.15.17:0/3065068049 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525801c070 con 0x7f52680febe0 2023-07-14T12:00:43.317 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.335+0000 7f526cfd9700 1 -- 172.21.15.17:0/3065068049 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5258009520 con 0x7f52680febe0 2023-07-14T12:00:43.317 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.335+0000 7f526cfd9700 1 -- 172.21.15.17:0/3065068049 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5268069f30 con 0x7f52680febe0 2023-07-14T12:00:43.317 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.335+0000 7f52577fe700 1 -- 172.21.15.17:0/3065068049 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5258005040 con 0x7f52680febe0 2023-07-14T12:00:43.317 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.335+0000 7f52577fe700 1 -- 172.21.15.17:0/3065068049 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52580038e0 con 0x7f52680febe0 2023-07-14T12:00:43.317 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.336+0000 7f526cfd9700 1 -- 172.21.15.17:0/3065068049 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5248000ff0 con 0x7f52680febe0 2023-07-14T12:00:43.319 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.338+0000 7f52577fe700 1 -- 172.21.15.17:0/3065068049 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5258020030 con 0x7f52680febe0 2023-07-14T12:00:43.320 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.340+0000 7f52577fe700 1 --2- 172.21.15.17:0/3065068049 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f52500a11d0 0x7f52500a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:43.321 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.340+0000 7f52577fe700 1 -- 172.21.15.17:0/3065068049 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f52580cdd20 con 0x7f52680febe0 2023-07-14T12:00:43.321 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.340+0000 7f5265d9b700 1 --2- 172.21.15.17:0/3065068049 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f52500a11d0 0x7f52500a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:43.321 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.340+0000 7f52577fe700 1 --2- 172.21.15.17:0/3065068049 >> [v2:172.21.15.97:6816/1965492859,v1:172.21.15.97:6817/1965492859] conn(0x7f52500ac0b0 0x7f52500ae540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:43.321 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.340+0000 7f52577fe700 1 -- 172.21.15.17:0/3065068049 --> [v2:172.21.15.97:6816/1965492859,v1:172.21.15.97:6817/1965492859] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f52500aec10 con 0x7f52500ac0b0 2023-07-14T12:00:43.321 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.340+0000 7f52577fe700 1 -- 172.21.15.17:0/3065068049 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52580ce130 con 0x7f52680febe0 2023-07-14T12:00:43.322 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.341+0000 7f5266d9d700 1 --2- 172.21.15.17:0/3065068049 >> [v2:172.21.15.97:6816/1965492859,v1:172.21.15.97:6817/1965492859] conn(0x7f52500ac0b0 0x7f52500ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:43.322 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.341+0000 7f5265d9b700 1 --2- 172.21.15.17:0/3065068049 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f52500a11d0 0x7f52500a3680 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f526819d8f0 tx=0x7f525c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:43.322 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.341+0000 7f5266d9d700 1 --2- 172.21.15.17:0/3065068049 >> [v2:172.21.15.97:6816/1965492859,v1:172.21.15.97:6817/1965492859] conn(0x7f52500ac0b0 0x7f52500ae540 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:43.322 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.342+0000 7f52577fe700 1 -- 172.21.15.17:0/3065068049 <== osd.8 v2:172.21.15.97:6816/1965492859 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f52500aec10 con 0x7f52500ac0b0 2023-07-14T12:00:43.340 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.359+0000 7f526cfd9700 1 -- 172.21.15.17:0/3065068049 --> [v2:172.21.15.97:6816/1965492859,v1:172.21.15.97:6817/1965492859] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5248002d70 con 0x7f52500ac0b0 2023-07-14T12:00:43.340 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.360+0000 7f52577fe700 1 -- 172.21.15.17:0/3065068049 <== osd.8 v2:172.21.15.97:6816/1965492859 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f5248002d70 con 0x7f52500ac0b0 2023-07-14T12:00:43.341 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.361+0000 7f526cfd9700 1 -- 172.21.15.17:0/3065068049 >> [v2:172.21.15.97:6816/1965492859,v1:172.21.15.97:6817/1965492859] conn(0x7f52500ac0b0 msgr2=0x7f52500ae540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:43.341 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.361+0000 7f526cfd9700 1 --2- 172.21.15.17:0/3065068049 >> [v2:172.21.15.97:6816/1965492859,v1:172.21.15.97:6817/1965492859] conn(0x7f52500ac0b0 0x7f52500ae540 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:43.341 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.361+0000 7f526cfd9700 1 -- 172.21.15.17:0/3065068049 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f52500a11d0 msgr2=0x7f52500a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:43.342 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.361+0000 7f526cfd9700 1 --2- 172.21.15.17:0/3065068049 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f52500a11d0 0x7f52500a3680 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f526819d8f0 tx=0x7f525c009260 comp rx=0 tx=0).stop 2023-07-14T12:00:43.342 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.361+0000 7f526cfd9700 1 -- 172.21.15.17:0/3065068049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52680febe0 msgr2=0x7f526819c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:43.342 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.361+0000 7f526cfd9700 1 --2- 172.21.15.17:0/3065068049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52680febe0 0x7f526819c2d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5258009810 tx=0x7f525800fa00 comp rx=0 tx=0).stop 2023-07-14T12:00:43.342 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.361+0000 7f5266d9d700 1 -- 172.21.15.17:0/3065068049 reap_dead start 2023-07-14T12:00:43.342 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.361+0000 7f526cfd9700 1 -- 172.21.15.17:0/3065068049 shutdown_connections 2023-07-14T12:00:43.342 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.361+0000 7f526cfd9700 1 -- 172.21.15.17:0/3065068049 >> 172.21.15.17:0/3065068049 conn(0x7f5268075b70 msgr2=0x7f5268101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:43.343 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.361+0000 7f526cfd9700 1 -- 172.21.15.17:0/3065068049 shutdown_connections 2023-07-14T12:00:43.343 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.361+0000 7f526cfd9700 1 -- 172.21.15.17:0/3065068049 wait complete. 2023-07-14T12:00:43.474 INFO:teuthology.orchestra.run.smithi017.stdout:146028888120 2023-07-14T12:00:43.475 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.4 2023-07-14T12:00:43.713 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.733+0000 7f5885d40700 1 -- 172.21.15.17:0/331779264 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5880104ea0 msgr2=0x7f588010ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:43.714 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.733+0000 7f5885d40700 1 --2- 172.21.15.17:0/331779264 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5880104ea0 0x7f588010ca20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f587800b0d0 tx=0x7f587800b3e0 comp rx=0 tx=0).stop 2023-07-14T12:00:43.714 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.733+0000 7f5885d40700 1 -- 172.21.15.17:0/331779264 shutdown_connections 2023-07-14T12:00:43.714 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.733+0000 7f5885d40700 1 --2- 172.21.15.17:0/331779264 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f58800ffe00 0x7f58801001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:43.714 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.733+0000 7f5885d40700 1 --2- 172.21.15.17:0/331779264 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5880104ea0 0x7f588010ca20 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:43.714 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.733+0000 7f5885d40700 1 --2- 172.21.15.17:0/331779264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58801007a0 0x7f5880104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:43.714 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.733+0000 7f5885d40700 1 -- 172.21.15.17:0/331779264 >> 172.21.15.17:0/331779264 conn(0x7f58800fb660 msgr2=0x7f58800fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:43.714 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.733+0000 7f5885d40700 1 -- 172.21.15.17:0/331779264 shutdown_connections 2023-07-14T12:00:43.715 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.733+0000 7f5885d40700 1 -- 172.21.15.17:0/331779264 wait complete. 2023-07-14T12:00:43.715 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.734+0000 7f5885d40700 1 Processor -- start 2023-07-14T12:00:43.715 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.735+0000 7f5885d40700 1 -- start start 2023-07-14T12:00:43.716 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.735+0000 7f5885d40700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f58800ffe00 0x7f5880195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:43.716 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.735+0000 7f5885d40700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58801007a0 0x7f58801962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:43.716 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.735+0000 7f5885d40700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5880104ea0 0x7f588019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:43.716 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.735+0000 7f5885d40700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588019ad20 con 0x7f5880104ea0 2023-07-14T12:00:43.717 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.735+0000 7f5885d40700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588019ae90 con 0x7f58800ffe00 2023-07-14T12:00:43.717 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.735+0000 7f5885d40700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588019b050 con 0x7f58801007a0 2023-07-14T12:00:43.718 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.736+0000 7f587f7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f58800ffe00 0x7f5880195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:43.718 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.736+0000 7f587effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58801007a0 0x7f58801962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:43.718 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.736+0000 7f587ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5880104ea0 0x7f588019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:43.718 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.736+0000 7f587effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58801007a0 0x7f58801962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.17:37772/0 (socket says 172.21.15.17:37772) 2023-07-14T12:00:43.718 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.736+0000 7f587effd700 1 -- 172.21.15.17:0/2349081852 learned_addr learned my addr 172.21.15.17:0/2349081852 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:00:43.718 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.736+0000 7f587ffff700 1 -- 172.21.15.17:0/2349081852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58801007a0 msgr2=0x7f58801962a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:43.719 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.736+0000 7f587ffff700 1 --2- 172.21.15.17:0/2349081852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58801007a0 0x7f58801962a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:43.719 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.736+0000 7f587ffff700 1 -- 172.21.15.17:0/2349081852 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f58800ffe00 msgr2=0x7f5880195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:43.719 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.736+0000 7f587ffff700 1 --2- 172.21.15.17:0/2349081852 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f58800ffe00 0x7f5880195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:43.719 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.736+0000 7f587ffff700 1 -- 172.21.15.17:0/2349081852 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5878009d00 con 0x7f5880104ea0 2023-07-14T12:00:43.719 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.736+0000 7f587effd700 1 --2- 172.21.15.17:0/2349081852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58801007a0 0x7f58801962a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T12:00:43.719 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.737+0000 7f587ffff700 1 --2- 172.21.15.17:0/2349081852 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5880104ea0 0x7f588019a710 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5878009fd0 tx=0x7f58780077f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:43.719 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.737+0000 7f587cff9700 1 -- 172.21.15.17:0/2349081852 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587800e040 con 0x7f5880104ea0 2023-07-14T12:00:43.720 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.737+0000 7f5885d40700 1 -- 172.21.15.17:0/2349081852 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f588019b2d0 con 0x7f5880104ea0 2023-07-14T12:00:43.720 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.737+0000 7f587cff9700 1 -- 172.21.15.17:0/2349081852 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f587800bcb0 con 0x7f5880104ea0 2023-07-14T12:00:43.720 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.737+0000 7f587cff9700 1 -- 172.21.15.17:0/2349081852 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5878005840 con 0x7f5880104ea0 2023-07-14T12:00:43.720 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.737+0000 7f5885d40700 1 -- 172.21.15.17:0/2349081852 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58801a2b90 con 0x7f5880104ea0 2023-07-14T12:00:43.720 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.738+0000 7f5885d40700 1 -- 172.21.15.17:0/2349081852 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f588004e210 con 0x7f5880104ea0 2023-07-14T12:00:43.721 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.741+0000 7f587cff9700 1 -- 172.21.15.17:0/2349081852 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5878004a50 con 0x7f5880104ea0 2023-07-14T12:00:43.723 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.742+0000 7f587cff9700 1 --2- 172.21.15.17:0/2349081852 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f58680a11d0 0x7f58680a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:43.723 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.742+0000 7f587cff9700 1 -- 172.21.15.17:0/2349081852 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f58780ce4c0 con 0x7f5880104ea0 2023-07-14T12:00:43.723 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.743+0000 7f587f7fe700 1 --2- 172.21.15.17:0/2349081852 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f58680a11d0 0x7f58680a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:43.724 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.743+0000 7f587cff9700 1 --2- 172.21.15.17:0/2349081852 >> [v2:172.21.15.93:6816/2596912392,v1:172.21.15.93:6817/2596912392] conn(0x7f58680ac1e0 0x7f58680ae670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:43.724 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.743+0000 7f587cff9700 1 -- 172.21.15.17:0/2349081852 --> [v2:172.21.15.93:6816/2596912392,v1:172.21.15.93:6817/2596912392] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f58680aed40 con 0x7f58680ac1e0 2023-07-14T12:00:43.724 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.743+0000 7f587cff9700 1 -- 172.21.15.17:0/2349081852 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58780ce870 con 0x7f5880104ea0 2023-07-14T12:00:43.724 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.743+0000 7f587effd700 1 --2- 172.21.15.17:0/2349081852 >> [v2:172.21.15.93:6816/2596912392,v1:172.21.15.93:6817/2596912392] conn(0x7f58680ac1e0 0x7f58680ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:43.725 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.744+0000 7f587f7fe700 1 --2- 172.21.15.17:0/2349081852 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f58680a11d0 0x7f58680a3680 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5870009c60 tx=0x7f5870009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:43.725 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.744+0000 7f587effd700 1 --2- 172.21.15.17:0/2349081852 >> [v2:172.21.15.93:6816/2596912392,v1:172.21.15.93:6817/2596912392] conn(0x7f58680ac1e0 0x7f58680ae670 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:43.727 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.746+0000 7f587cff9700 1 -- 172.21.15.17:0/2349081852 <== osd.7 v2:172.21.15.93:6816/2596912392 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f58680aed40 con 0x7f58680ac1e0 2023-07-14T12:00:43.747 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.766+0000 7f5885d40700 1 -- 172.21.15.17:0/2349081852 --> [v2:172.21.15.93:6816/2596912392,v1:172.21.15.93:6817/2596912392] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f588010fd90 con 0x7f58680ac1e0 2023-07-14T12:00:43.747 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.767+0000 7f587cff9700 1 -- 172.21.15.17:0/2349081852 <== osd.7 v2:172.21.15.93:6816/2596912392 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f5874009dc0 con 0x7f58680ac1e0 2023-07-14T12:00:43.748 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.768+0000 7f5885d40700 1 -- 172.21.15.17:0/2349081852 >> [v2:172.21.15.93:6816/2596912392,v1:172.21.15.93:6817/2596912392] conn(0x7f58680ac1e0 msgr2=0x7f58680ae670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:43.748 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.768+0000 7f5885d40700 1 --2- 172.21.15.17:0/2349081852 >> [v2:172.21.15.93:6816/2596912392,v1:172.21.15.93:6817/2596912392] conn(0x7f58680ac1e0 0x7f58680ae670 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:43.748 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.768+0000 7f5885d40700 1 -- 172.21.15.17:0/2349081852 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f58680a11d0 msgr2=0x7f58680a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:43.748 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.768+0000 7f5885d40700 1 --2- 172.21.15.17:0/2349081852 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f58680a11d0 0x7f58680a3680 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5870009c60 tx=0x7f5870009260 comp rx=0 tx=0).stop 2023-07-14T12:00:43.749 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.768+0000 7f5885d40700 1 -- 172.21.15.17:0/2349081852 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5880104ea0 msgr2=0x7f588019a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:43.749 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.768+0000 7f5885d40700 1 --2- 172.21.15.17:0/2349081852 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5880104ea0 0x7f588019a710 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5878009fd0 tx=0x7f58780077f0 comp rx=0 tx=0).stop 2023-07-14T12:00:43.749 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.768+0000 7f587ffff700 1 -- 172.21.15.17:0/2349081852 reap_dead start 2023-07-14T12:00:43.749 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.768+0000 7f5885d40700 1 -- 172.21.15.17:0/2349081852 shutdown_connections 2023-07-14T12:00:43.750 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.768+0000 7f5885d40700 1 -- 172.21.15.17:0/2349081852 >> 172.21.15.17:0/2349081852 conn(0x7f58800fb660 msgr2=0x7f58800fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:43.750 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.768+0000 7f5885d40700 1 -- 172.21.15.17:0/2349081852 shutdown_connections 2023-07-14T12:00:43.750 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.768+0000 7f5885d40700 1 -- 172.21.15.17:0/2349081852 wait complete. 2023-07-14T12:00:44.159 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.948+0000 7fe8151ac700 1 -- 172.21.15.17:0/2884764603 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe810104ea0 msgr2=0x7fe81010ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:44.159 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.948+0000 7fe8151ac700 1 --2- 172.21.15.17:0/2884764603 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe810104ea0 0x7fe81010ca20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe808009fc0 tx=0x7fe80800b330 comp rx=0 tx=0).stop 2023-07-14T12:00:44.159 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.949+0000 7fe8151ac700 1 -- 172.21.15.17:0/2884764603 shutdown_connections 2023-07-14T12:00:44.159 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.949+0000 7fe8151ac700 1 --2- 172.21.15.17:0/2884764603 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8100ffe00 0x7fe8101001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:44.159 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.949+0000 7fe8151ac700 1 --2- 172.21.15.17:0/2884764603 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe810104ea0 0x7fe81010ca20 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:44.160 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.949+0000 7fe8151ac700 1 --2- 172.21.15.17:0/2884764603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8101007a0 0x7fe810104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:44.160 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.949+0000 7fe8151ac700 1 -- 172.21.15.17:0/2884764603 >> 172.21.15.17:0/2884764603 conn(0x7fe8100fb660 msgr2=0x7fe8100fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:44.160 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.949+0000 7fe8151ac700 1 -- 172.21.15.17:0/2884764603 shutdown_connections 2023-07-14T12:00:44.160 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.949+0000 7fe8151ac700 1 -- 172.21.15.17:0/2884764603 wait complete. 2023-07-14T12:00:44.160 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.950+0000 7fe8151ac700 1 Processor -- start 2023-07-14T12:00:44.160 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.950+0000 7fe8151ac700 1 -- start start 2023-07-14T12:00:44.160 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.950+0000 7fe8151ac700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8100ffe00 0x7fe81006f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:44.160 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.950+0000 7fe8151ac700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8101007a0 0x7fe81006fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:44.161 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.950+0000 7fe8151ac700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe810104ea0 0x7fe81006a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:44.161 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.950+0000 7fe8151ac700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81006af20 con 0x7fe8101007a0 2023-07-14T12:00:44.161 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.950+0000 7fe8151ac700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81006b090 con 0x7fe810104ea0 2023-07-14T12:00:44.161 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.950+0000 7fe8151ac700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81006b250 con 0x7fe8100ffe00 2023-07-14T12:00:44.161 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.951+0000 7fe80e59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8101007a0 0x7fe81006fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:44.161 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.951+0000 7fe80ed9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8100ffe00 0x7fe81006f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:44.161 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.951+0000 7fe80e59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8101007a0 0x7fe81006fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37916/0 (socket says 172.21.15.17:37916) 2023-07-14T12:00:44.162 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.951+0000 7fe80ed9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8100ffe00 0x7fe81006f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.17:37798/0 (socket says 172.21.15.17:37798) 2023-07-14T12:00:44.162 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.951+0000 7fe80e59c700 1 -- 172.21.15.17:0/3855659740 learned_addr learned my addr 172.21.15.17:0/3855659740 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:00:44.162 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.951+0000 7fe80f59e700 1 --2- 172.21.15.17:0/3855659740 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe810104ea0 0x7fe81006a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:44.162 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.951+0000 7fe80f59e700 1 -- 172.21.15.17:0/3855659740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8100ffe00 msgr2=0x7fe81006f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:44.162 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.951+0000 7fe80f59e700 1 --2- 172.21.15.17:0/3855659740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8100ffe00 0x7fe81006f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:44.162 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.951+0000 7fe80f59e700 1 -- 172.21.15.17:0/3855659740 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8101007a0 msgr2=0x7fe81006fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:44.162 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.951+0000 7fe80f59e700 1 --2- 172.21.15.17:0/3855659740 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8101007a0 0x7fe81006fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:44.163 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.951+0000 7fe80f59e700 1 -- 172.21.15.17:0/3855659740 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe808009c40 con 0x7fe810104ea0 2023-07-14T12:00:44.163 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.951+0000 7fe80ed9d700 1 --2- 172.21.15.17:0/3855659740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8100ffe00 0x7fe81006f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T12:00:44.163 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.951+0000 7fe80e59c700 1 --2- 172.21.15.17:0/3855659740 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8101007a0 0x7fe81006fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T12:00:44.163 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.952+0000 7fe80f59e700 1 --2- 172.21.15.17:0/3855659740 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe810104ea0 0x7fe81006a910 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe808015000 tx=0x7fe8080093b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:44.163 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.952+0000 7fe7f7fff700 1 -- 172.21.15.17:0/3855659740 <== mon.2 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80800e040 con 0x7fe810104ea0 2023-07-14T12:00:44.164 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.952+0000 7fe8151ac700 1 -- 172.21.15.17:0/3855659740 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe81006b4d0 con 0x7fe810104ea0 2023-07-14T12:00:44.164 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.952+0000 7fe8151ac700 1 -- 172.21.15.17:0/3855659740 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe81006e650 con 0x7fe810104ea0 2023-07-14T12:00:44.164 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.952+0000 7fe7f7fff700 1 -- 172.21.15.17:0/3855659740 <== mon.2 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe808009ac0 con 0x7fe810104ea0 2023-07-14T12:00:44.164 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.952+0000 7fe7f7fff700 1 -- 172.21.15.17:0/3855659740 <== mon.2 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80801dd60 con 0x7fe810104ea0 2023-07-14T12:00:44.164 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.953+0000 7fe8151ac700 1 -- 172.21.15.17:0/3855659740 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe7fc000ff0 con 0x7fe810104ea0 2023-07-14T12:00:44.164 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.956+0000 7fe7f7fff700 1 -- 172.21.15.17:0/3855659740 <== mon.2 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe80801d470 con 0x7fe810104ea0 2023-07-14T12:00:44.165 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.957+0000 7fe7f7fff700 1 --2- 172.21.15.17:0/3855659740 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fe7f80a1150 0x7fe7f80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:44.165 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.957+0000 7fe7f7fff700 1 -- 172.21.15.17:0/3855659740 <== mon.2 v2:172.21.15.93:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe8080ce6f0 con 0x7fe810104ea0 2023-07-14T12:00:44.165 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.957+0000 7fe80ed9d700 1 --2- 172.21.15.17:0/3855659740 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fe7f80a1150 0x7fe7f80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:44.165 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.958+0000 7fe7f7fff700 1 --2- 172.21.15.17:0/3855659740 >> [v2:172.21.15.17:6826/2512645700,v1:172.21.15.17:6827/2512645700] conn(0x7fe7f80ac160 0x7fe7f80ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:44.165 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.958+0000 7fe7f7fff700 1 -- 172.21.15.17:0/3855659740 --> [v2:172.21.15.17:6826/2512645700,v1:172.21.15.17:6827/2512645700] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe7f80aecc0 con 0x7fe7f80ac160 2023-07-14T12:00:44.166 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.958+0000 7fe7f7fff700 1 -- 172.21.15.17:0/3855659740 <== mon.2 v2:172.21.15.93:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8080cea60 con 0x7fe810104ea0 2023-07-14T12:00:44.166 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.958+0000 7fe80e59c700 1 --2- 172.21.15.17:0/3855659740 >> [v2:172.21.15.17:6826/2512645700,v1:172.21.15.17:6827/2512645700] conn(0x7fe7f80ac160 0x7fe7f80ae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:44.166 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.958+0000 7fe80ed9d700 1 --2- 172.21.15.17:0/3855659740 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fe7f80a1150 0x7fe7f80a3600 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe800011490 tx=0x7fe800009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:44.166 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.959+0000 7fe80e59c700 1 --2- 172.21.15.17:0/3855659740 >> [v2:172.21.15.17:6826/2512645700,v1:172.21.15.17:6827/2512645700] conn(0x7fe7f80ac160 0x7fe7f80ae5f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:44.166 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.960+0000 7fe7f7fff700 1 -- 172.21.15.17:0/3855659740 <== osd.9 v2:172.21.15.17:6826/2512645700 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fe7f80aecc0 con 0x7fe7f80ac160 2023-07-14T12:00:44.166 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.977+0000 7fe8151ac700 1 -- 172.21.15.17:0/3855659740 --> [v2:172.21.15.17:6826/2512645700,v1:172.21.15.17:6827/2512645700] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe7fc002d70 con 0x7fe7f80ac160 2023-07-14T12:00:44.166 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.977+0000 7fe7f7fff700 1 -- 172.21.15.17:0/3855659740 <== osd.9 v2:172.21.15.17:6826/2512645700 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe7fc002d70 con 0x7fe7f80ac160 2023-07-14T12:00:44.167 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.978+0000 7fe8151ac700 1 -- 172.21.15.17:0/3855659740 >> [v2:172.21.15.17:6826/2512645700,v1:172.21.15.17:6827/2512645700] conn(0x7fe7f80ac160 msgr2=0x7fe7f80ae5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:44.167 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.978+0000 7fe8151ac700 1 --2- 172.21.15.17:0/3855659740 >> [v2:172.21.15.17:6826/2512645700,v1:172.21.15.17:6827/2512645700] conn(0x7fe7f80ac160 0x7fe7f80ae5f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:44.167 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.978+0000 7fe8151ac700 1 -- 172.21.15.17:0/3855659740 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fe7f80a1150 msgr2=0x7fe7f80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:44.167 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.978+0000 7fe8151ac700 1 --2- 172.21.15.17:0/3855659740 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fe7f80a1150 0x7fe7f80a3600 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe800011490 tx=0x7fe800009260 comp rx=0 tx=0).stop 2023-07-14T12:00:44.167 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.978+0000 7fe8151ac700 1 -- 172.21.15.17:0/3855659740 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe810104ea0 msgr2=0x7fe81006a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:44.167 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.978+0000 7fe8151ac700 1 --2- 172.21.15.17:0/3855659740 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe810104ea0 0x7fe81006a910 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe808015000 tx=0x7fe8080093b0 comp rx=0 tx=0).stop 2023-07-14T12:00:44.167 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.978+0000 7fe80f59e700 1 -- 172.21.15.17:0/3855659740 reap_dead start 2023-07-14T12:00:44.168 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.978+0000 7fe8151ac700 1 -- 172.21.15.17:0/3855659740 shutdown_connections 2023-07-14T12:00:44.168 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.978+0000 7fe8151ac700 1 -- 172.21.15.17:0/3855659740 >> 172.21.15.17:0/3855659740 conn(0x7fe8100fb660 msgr2=0x7fe8100fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:44.168 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.978+0000 7fe8151ac700 1 -- 172.21.15.17:0/3855659740 shutdown_connections 2023-07-14T12:00:44.168 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:43.978+0000 7fe8151ac700 1 -- 172.21.15.17:0/3855659740 wait complete. 2023-07-14T12:00:44.359 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.059+0000 7fec001b9700 1 -- 172.21.15.17:0/669851661 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febf81007a0 msgr2=0x7febf8104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:44.360 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.059+0000 7fec001b9700 1 --2- 172.21.15.17:0/669851661 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febf81007a0 0x7febf8104770 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7febe80099c0 tx=0x7febe8009cd0 comp rx=0 tx=0).stop 2023-07-14T12:00:44.360 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.059+0000 7fec001b9700 1 -- 172.21.15.17:0/669851661 shutdown_connections 2023-07-14T12:00:44.360 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.059+0000 7fec001b9700 1 --2- 172.21.15.17:0/669851661 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7febf8104ea0 0x7febf810ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:44.360 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.059+0000 7fec001b9700 1 --2- 172.21.15.17:0/669851661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febf80ffe00 0x7febf81001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:44.360 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.059+0000 7fec001b9700 1 --2- 172.21.15.17:0/669851661 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febf81007a0 0x7febf8104770 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:44.360 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.059+0000 7fec001b9700 1 -- 172.21.15.17:0/669851661 >> 172.21.15.17:0/669851661 conn(0x7febf80fb660 msgr2=0x7febf80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:44.361 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.059+0000 7fec001b9700 1 -- 172.21.15.17:0/669851661 shutdown_connections 2023-07-14T12:00:44.361 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.059+0000 7fec001b9700 1 -- 172.21.15.17:0/669851661 wait complete. 2023-07-14T12:00:44.361 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.060+0000 7fec001b9700 1 Processor -- start 2023-07-14T12:00:44.361 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.060+0000 7fec001b9700 1 -- start start 2023-07-14T12:00:44.361 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.061+0000 7fec001b9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febf80ffe00 0x7febf8197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:44.361 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.061+0000 7fec001b9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febf81007a0 0x7febf8198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:44.361 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.061+0000 7fec001b9700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7febf8104ea0 0x7febf819c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:44.362 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.061+0000 7fec001b9700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf819cec0 con 0x7febf80ffe00 2023-07-14T12:00:44.362 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.061+0000 7fec001b9700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf819d030 con 0x7febf8104ea0 2023-07-14T12:00:44.362 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.061+0000 7fec001b9700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf819d1f0 con 0x7febf81007a0 2023-07-14T12:00:44.362 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.061+0000 7febfdf55700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febf80ffe00 0x7febf8197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:44.362 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.061+0000 7febfdf55700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febf80ffe00 0x7febf8197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37926/0 (socket says 172.21.15.17:37926) 2023-07-14T12:00:44.362 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.061+0000 7febfdf55700 1 -- 172.21.15.17:0/2813018873 learned_addr learned my addr 172.21.15.17:0/2813018873 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:00:44.362 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.061+0000 7febfd754700 1 --2- 172.21.15.17:0/2813018873 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febf81007a0 0x7febf8198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:44.362 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.061+0000 7febfe756700 1 --2- 172.21.15.17:0/2813018873 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7febf8104ea0 0x7febf819c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:44.363 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.062+0000 7febfdf55700 1 -- 172.21.15.17:0/2813018873 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febf81007a0 msgr2=0x7febf8198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:44.363 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.062+0000 7febfdf55700 1 --2- 172.21.15.17:0/2813018873 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febf81007a0 0x7febf8198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:44.363 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.062+0000 7febfdf55700 1 -- 172.21.15.17:0/2813018873 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7febf8104ea0 msgr2=0x7febf819c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:44.363 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.062+0000 7febfdf55700 1 --2- 172.21.15.17:0/2813018873 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7febf8104ea0 0x7febf819c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:44.363 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.062+0000 7febfdf55700 1 -- 172.21.15.17:0/2813018873 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf4009520 con 0x7febf80ffe00 2023-07-14T12:00:44.363 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.062+0000 7febfe756700 1 --2- 172.21.15.17:0/2813018873 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7febf8104ea0 0x7febf819c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T12:00:44.363 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.062+0000 7febfdf55700 1 --2- 172.21.15.17:0/2813018873 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febf80ffe00 0x7febf8197f00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7febf400eb30 tx=0x7febf400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:44.364 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.062+0000 7febeeffd700 1 -- 172.21.15.17:0/2813018873 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf4009880 con 0x7febf80ffe00 2023-07-14T12:00:44.364 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.062+0000 7fec001b9700 1 -- 172.21.15.17:0/2813018873 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febe80095f0 con 0x7febf80ffe00 2023-07-14T12:00:44.364 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.062+0000 7febeeffd700 1 -- 172.21.15.17:0/2813018873 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febf40044e0 con 0x7febf80ffe00 2023-07-14T12:00:44.364 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.062+0000 7febeeffd700 1 -- 172.21.15.17:0/2813018873 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf4003e90 con 0x7febf80ffe00 2023-07-14T12:00:44.364 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.062+0000 7fec001b9700 1 -- 172.21.15.17:0/2813018873 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febf81a4b60 con 0x7febf80ffe00 2023-07-14T12:00:44.364 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.063+0000 7fec001b9700 1 -- 172.21.15.17:0/2813018873 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7febf804e210 con 0x7febf80ffe00 2023-07-14T12:00:44.364 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.066+0000 7febeeffd700 1 -- 172.21.15.17:0/2813018873 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7febf4004000 con 0x7febf80ffe00 2023-07-14T12:00:44.365 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.067+0000 7febeeffd700 1 --2- 172.21.15.17:0/2813018873 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7febe40a11d0 0x7febe40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:44.365 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.067+0000 7febeeffd700 1 -- 172.21.15.17:0/2813018873 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7febf4014070 con 0x7febf80ffe00 2023-07-14T12:00:44.365 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.067+0000 7febfd754700 1 --2- 172.21.15.17:0/2813018873 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7febe40a11d0 0x7febe40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:44.365 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.068+0000 7febeeffd700 1 --2- 172.21.15.17:0/2813018873 >> [v2:172.21.15.93:6824/264429558,v1:172.21.15.93:6825/264429558] conn(0x7febe40ac070 0x7febe40ae500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:44.365 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.068+0000 7febeeffd700 1 -- 172.21.15.17:0/2813018873 --> [v2:172.21.15.93:6824/264429558,v1:172.21.15.93:6825/264429558] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7febe40aebd0 con 0x7febe40ac070 2023-07-14T12:00:44.365 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.068+0000 7febeeffd700 1 -- 172.21.15.17:0/2813018873 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7febf40cd440 con 0x7febf80ffe00 2023-07-14T12:00:44.365 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.068+0000 7febfe756700 1 --2- 172.21.15.17:0/2813018873 >> [v2:172.21.15.93:6824/264429558,v1:172.21.15.93:6825/264429558] conn(0x7febe40ac070 0x7febe40ae500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:44.366 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.068+0000 7febfd754700 1 --2- 172.21.15.17:0/2813018873 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7febe40a11d0 0x7febe40a3680 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7febe8009fd0 tx=0x7febe8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:44.366 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.069+0000 7febfe756700 1 --2- 172.21.15.17:0/2813018873 >> [v2:172.21.15.93:6824/264429558,v1:172.21.15.93:6825/264429558] conn(0x7febe40ac070 0x7febe40ae500 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:44.367 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.069+0000 7febeeffd700 1 -- 172.21.15.17:0/2813018873 <== osd.10 v2:172.21.15.93:6824/264429558 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7febe40aebd0 con 0x7febe40ac070 2023-07-14T12:00:44.367 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.087+0000 7fec001b9700 1 -- 172.21.15.17:0/2813018873 --> [v2:172.21.15.93:6824/264429558,v1:172.21.15.93:6825/264429558] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7febf810fd90 con 0x7febe40ac070 2023-07-14T12:00:44.367 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.088+0000 7febeeffd700 1 -- 172.21.15.17:0/2813018873 <== osd.10 v2:172.21.15.93:6824/264429558 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7febf000f040 con 0x7febe40ac070 2023-07-14T12:00:44.367 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.089+0000 7fec001b9700 1 -- 172.21.15.17:0/2813018873 >> [v2:172.21.15.93:6824/264429558,v1:172.21.15.93:6825/264429558] conn(0x7febe40ac070 msgr2=0x7febe40ae500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:44.367 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.089+0000 7fec001b9700 1 --2- 172.21.15.17:0/2813018873 >> [v2:172.21.15.93:6824/264429558,v1:172.21.15.93:6825/264429558] conn(0x7febe40ac070 0x7febe40ae500 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:44.368 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.089+0000 7fec001b9700 1 -- 172.21.15.17:0/2813018873 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7febe40a11d0 msgr2=0x7febe40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:44.368 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.089+0000 7fec001b9700 1 --2- 172.21.15.17:0/2813018873 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7febe40a11d0 0x7febe40a3680 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7febe8009fd0 tx=0x7febe8005fb0 comp rx=0 tx=0).stop 2023-07-14T12:00:44.368 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.089+0000 7fec001b9700 1 -- 172.21.15.17:0/2813018873 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febf80ffe00 msgr2=0x7febf8197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:44.368 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.089+0000 7fec001b9700 1 --2- 172.21.15.17:0/2813018873 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febf80ffe00 0x7febf8197f00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7febf400eb30 tx=0x7febf400ee40 comp rx=0 tx=0).stop 2023-07-14T12:00:44.368 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.089+0000 7febfe756700 1 -- 172.21.15.17:0/2813018873 reap_dead start 2023-07-14T12:00:44.368 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.089+0000 7fec001b9700 1 -- 172.21.15.17:0/2813018873 shutdown_connections 2023-07-14T12:00:44.369 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.089+0000 7fec001b9700 1 -- 172.21.15.17:0/2813018873 >> 172.21.15.17:0/2813018873 conn(0x7febf80fb660 msgr2=0x7febf80fc2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:44.369 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.090+0000 7fec001b9700 1 -- 172.21.15.17:0/2813018873 shutdown_connections 2023-07-14T12:00:44.369 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.090+0000 7fec001b9700 1 -- 172.21.15.17:0/2813018873 wait complete. 2023-07-14T12:00:44.476 INFO:teuthology.orchestra.run.smithi017.stdout:253403070502 2023-07-14T12:00:44.477 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.8 2023-07-14T12:00:44.545 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:44 smithi017 ceph-mon[126339]: pgmap v309: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-07-14T12:00:44.860 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:44 smithi093 ceph-mon[128922]: pgmap v309: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-07-14T12:00:44.884 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:44 smithi097 ceph-mon[129313]: pgmap v309: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-07-14T12:00:45.025 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.246+0000 7f298fd57700 1 -- 172.21.15.17:0/1243666790 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f29880731c0 msgr2=0x7f298810bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:45.025 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.246+0000 7f298fd57700 1 --2- 172.21.15.17:0/1243666790 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f29880731c0 0x7f298810bea0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2978009990 tx=0x7f2978009ca0 comp rx=0 tx=0).stop 2023-07-14T12:00:45.026 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.246+0000 7f298fd57700 1 -- 172.21.15.17:0/1243666790 shutdown_connections 2023-07-14T12:00:45.026 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.246+0000 7f298fd57700 1 --2- 172.21.15.17:0/1243666790 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f29880728b0 0x7f2988072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:45.026 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.246+0000 7f298fd57700 1 --2- 172.21.15.17:0/1243666790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f298810c3e0 0x7f298810e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:45.026 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.246+0000 7f298fd57700 1 --2- 172.21.15.17:0/1243666790 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f29880731c0 0x7f298810bea0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:45.026 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.246+0000 7f298fd57700 1 -- 172.21.15.17:0/1243666790 >> 172.21.15.17:0/1243666790 conn(0x7f29880fb660 msgr2=0x7f29880fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:45.026 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.247+0000 7f298fd57700 1 -- 172.21.15.17:0/1243666790 shutdown_connections 2023-07-14T12:00:45.026 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.247+0000 7f298fd57700 1 -- 172.21.15.17:0/1243666790 wait complete. 2023-07-14T12:00:45.027 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.248+0000 7f298fd57700 1 Processor -- start 2023-07-14T12:00:45.027 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.248+0000 7f298fd57700 1 -- start start 2023-07-14T12:00:45.027 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.248+0000 7f298fd57700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f29880728b0 0x7f2988197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:45.027 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.248+0000 7f298fd57700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f29880731c0 0x7f2988198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:45.027 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.248+0000 7f298fd57700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f298810c3e0 0x7f298819c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:45.027 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.248+0000 7f298fd57700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298819ce40 con 0x7f29880731c0 2023-07-14T12:00:45.027 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.248+0000 7f298fd57700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298819cfb0 con 0x7f29880728b0 2023-07-14T12:00:45.027 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.248+0000 7f298fd57700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298819d170 con 0x7f298810c3e0 2023-07-14T12:00:45.028 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.248+0000 7f298d2f2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f29880731c0 0x7f2988198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:45.028 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.249+0000 7f298d2f2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f29880731c0 0x7f2988198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37930/0 (socket says 172.21.15.17:37930) 2023-07-14T12:00:45.028 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.249+0000 7f298d2f2700 1 -- 172.21.15.17:0/1659912479 learned_addr learned my addr 172.21.15.17:0/1659912479 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:00:45.028 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.249+0000 7f298daf3700 1 --2- 172.21.15.17:0/1659912479 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f29880728b0 0x7f2988197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:45.028 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.249+0000 7f298e2f4700 1 --2- 172.21.15.17:0/1659912479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f298810c3e0 0x7f298819c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:45.028 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.249+0000 7f298d2f2700 1 -- 172.21.15.17:0/1659912479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f298810c3e0 msgr2=0x7f298819c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:45.028 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.249+0000 7f298d2f2700 1 --2- 172.21.15.17:0/1659912479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f298810c3e0 0x7f298819c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:45.029 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.249+0000 7f298d2f2700 1 -- 172.21.15.17:0/1659912479 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f29880728b0 msgr2=0x7f2988197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:45.029 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.249+0000 7f298d2f2700 1 --2- 172.21.15.17:0/1659912479 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f29880728b0 0x7f2988197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:45.029 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.249+0000 7f298d2f2700 1 -- 172.21.15.17:0/1659912479 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2984009520 con 0x7f29880731c0 2023-07-14T12:00:45.029 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.249+0000 7f298d2f2700 1 --2- 172.21.15.17:0/1659912479 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f29880731c0 0x7f2988198330 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2978005910 tx=0x7f2978005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:45.029 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.249+0000 7f297effd700 1 -- 172.21.15.17:0/1659912479 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297801d070 con 0x7f29880731c0 2023-07-14T12:00:45.029 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.249+0000 7f298fd57700 1 -- 172.21.15.17:0/1659912479 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29780095f0 con 0x7f29880731c0 2023-07-14T12:00:45.029 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.249+0000 7f297effd700 1 -- 172.21.15.17:0/1659912479 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2978022470 con 0x7f29880731c0 2023-07-14T12:00:45.029 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.249+0000 7f297effd700 1 -- 172.21.15.17:0/1659912479 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2978017650 con 0x7f29880731c0 2023-07-14T12:00:45.030 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.250+0000 7f298fd57700 1 -- 172.21.15.17:0/1659912479 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29881a4a50 con 0x7f29880731c0 2023-07-14T12:00:45.030 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.250+0000 7f298fd57700 1 -- 172.21.15.17:0/1659912479 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f298804e210 con 0x7f29880731c0 2023-07-14T12:00:45.030 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.253+0000 7f297effd700 1 -- 172.21.15.17:0/1659912479 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f29780225e0 con 0x7f29880731c0 2023-07-14T12:00:45.030 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.254+0000 7f297effd700 1 --2- 172.21.15.17:0/1659912479 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f29740a1150 0x7f29740a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:45.030 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.254+0000 7f297effd700 1 -- 172.21.15.17:0/1659912479 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f29780cea20 con 0x7f29880731c0 2023-07-14T12:00:45.031 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.254+0000 7f298daf3700 1 --2- 172.21.15.17:0/1659912479 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f29740a1150 0x7f29740a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:45.031 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.255+0000 7f297effd700 1 --2- 172.21.15.17:0/1659912479 >> [v2:172.21.15.17:6818/50486581,v1:172.21.15.17:6819/50486581] conn(0x7f29740ac160 0x7f29740ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:45.031 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.255+0000 7f297effd700 1 -- 172.21.15.17:0/1659912479 --> [v2:172.21.15.17:6818/50486581,v1:172.21.15.17:6819/50486581] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f29740aecc0 con 0x7f29740ac160 2023-07-14T12:00:45.031 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.255+0000 7f297effd700 1 -- 172.21.15.17:0/1659912479 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29780ced90 con 0x7f29880731c0 2023-07-14T12:00:45.031 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.255+0000 7f298e2f4700 1 --2- 172.21.15.17:0/1659912479 >> [v2:172.21.15.17:6818/50486581,v1:172.21.15.17:6819/50486581] conn(0x7f29740ac160 0x7f29740ae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:45.031 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.255+0000 7f298daf3700 1 --2- 172.21.15.17:0/1659912479 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f29740a1150 0x7f29740a3600 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2984009df0 tx=0x7f2984009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:45.031 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.256+0000 7f298e2f4700 1 --2- 172.21.15.17:0/1659912479 >> [v2:172.21.15.17:6818/50486581,v1:172.21.15.17:6819/50486581] conn(0x7f29740ac160 0x7f29740ae5f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:45.032 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.257+0000 7f297effd700 1 -- 172.21.15.17:0/1659912479 <== osd.6 v2:172.21.15.17:6818/50486581 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f29740aecc0 con 0x7f29740ac160 2023-07-14T12:00:45.032 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.278+0000 7f298fd57700 1 -- 172.21.15.17:0/1659912479 --> [v2:172.21.15.17:6818/50486581,v1:172.21.15.17:6819/50486581] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2988111f80 con 0x7f29740ac160 2023-07-14T12:00:45.032 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.278+0000 7f297effd700 1 -- 172.21.15.17:0/1659912479 <== osd.6 v2:172.21.15.17:6818/50486581 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f298000f040 con 0x7f29740ac160 2023-07-14T12:00:45.032 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.279+0000 7f298fd57700 1 -- 172.21.15.17:0/1659912479 >> [v2:172.21.15.17:6818/50486581,v1:172.21.15.17:6819/50486581] conn(0x7f29740ac160 msgr2=0x7f29740ae5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:45.032 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.279+0000 7f298fd57700 1 --2- 172.21.15.17:0/1659912479 >> [v2:172.21.15.17:6818/50486581,v1:172.21.15.17:6819/50486581] conn(0x7f29740ac160 0x7f29740ae5f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:45.032 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.279+0000 7f298fd57700 1 -- 172.21.15.17:0/1659912479 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f29740a1150 msgr2=0x7f29740a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:45.032 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.279+0000 7f298fd57700 1 --2- 172.21.15.17:0/1659912479 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f29740a1150 0x7f29740a3600 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2984009df0 tx=0x7f2984009260 comp rx=0 tx=0).stop 2023-07-14T12:00:45.033 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.279+0000 7f298fd57700 1 -- 172.21.15.17:0/1659912479 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f29880731c0 msgr2=0x7f2988198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:45.033 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.279+0000 7f298fd57700 1 --2- 172.21.15.17:0/1659912479 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f29880731c0 0x7f2988198330 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2978005910 tx=0x7f2978005720 comp rx=0 tx=0).stop 2023-07-14T12:00:45.033 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.279+0000 7f298e2f4700 1 -- 172.21.15.17:0/1659912479 reap_dead start 2023-07-14T12:00:45.033 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.279+0000 7f298fd57700 1 -- 172.21.15.17:0/1659912479 shutdown_connections 2023-07-14T12:00:45.033 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.279+0000 7f298fd57700 1 -- 172.21.15.17:0/1659912479 >> 172.21.15.17:0/1659912479 conn(0x7f29880fb660 msgr2=0x7f298810e480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:45.033 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.279+0000 7f298fd57700 1 -- 172.21.15.17:0/1659912479 shutdown_connections 2023-07-14T12:00:45.033 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:44.279+0000 7f298fd57700 1 -- 172.21.15.17:0/1659912479 wait complete. 2023-07-14T12:00:45.333 INFO:teuthology.orchestra.run.smithi017.stdout:223338299435 2023-07-14T12:00:45.333 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.7 2023-07-14T12:00:45.392 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.389+0000 7f961aba0700 1 -- 172.21.15.17:0/1192132190 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9614101700 msgr2=0x7f9614109c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:45.392 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.389+0000 7f961aba0700 1 --2- 172.21.15.17:0/1192132190 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9614101700 0x7f9614109c00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f96000098f0 tx=0x7f9600009c00 comp rx=0 tx=0).stop 2023-07-14T12:00:45.392 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.390+0000 7f961aba0700 1 -- 172.21.15.17:0/1192132190 shutdown_connections 2023-07-14T12:00:45.392 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.390+0000 7f961aba0700 1 --2- 172.21.15.17:0/1192132190 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f961410a140 0x7f961410c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:45.393 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.390+0000 7f961aba0700 1 --2- 172.21.15.17:0/1192132190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9614100df0 0x7f96141011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:45.393 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.390+0000 7f961aba0700 1 --2- 172.21.15.17:0/1192132190 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9614101700 0x7f9614109c00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:45.393 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.390+0000 7f961aba0700 1 -- 172.21.15.17:0/1192132190 >> 172.21.15.17:0/1192132190 conn(0x7f9614075b70 msgr2=0x7f9614075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:45.393 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.390+0000 7f961aba0700 1 -- 172.21.15.17:0/1192132190 shutdown_connections 2023-07-14T12:00:45.393 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.390+0000 7f961aba0700 1 -- 172.21.15.17:0/1192132190 wait complete. 2023-07-14T12:00:45.393 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.391+0000 7f961aba0700 1 Processor -- start 2023-07-14T12:00:45.393 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.392+0000 7f961aba0700 1 -- start start 2023-07-14T12:00:45.393 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.392+0000 7f961aba0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9614100df0 0x7f96140fea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:45.394 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.392+0000 7f961aba0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9614101700 0x7f96140fef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:45.394 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.392+0000 7f961aba0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f961410a140 0x7f96140ff510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:45.394 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.392+0000 7f961aba0700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96141a4180 con 0x7f9614101700 2023-07-14T12:00:45.394 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.392+0000 7f961aba0700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96141a42f0 con 0x7f9614100df0 2023-07-14T12:00:45.394 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.392+0000 7f961aba0700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96141a44b0 con 0x7f961410a140 2023-07-14T12:00:45.394 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.393+0000 7f961893c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9614100df0 0x7f96140fea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:45.394 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.393+0000 7f9613fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9614101700 0x7f96140fef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:45.395 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.393+0000 7f961893c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9614100df0 0x7f96140fea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.17:39516/0 (socket says 172.21.15.17:39516) 2023-07-14T12:00:45.395 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.393+0000 7f961893c700 1 -- 172.21.15.17:0/407894625 learned_addr learned my addr 172.21.15.17:0/407894625 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:00:45.395 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.393+0000 7f961913d700 1 --2- 172.21.15.17:0/407894625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f961410a140 0x7f96140ff510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:45.395 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.393+0000 7f961893c700 1 -- 172.21.15.17:0/407894625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f961410a140 msgr2=0x7f96140ff510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:45.395 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.393+0000 7f961893c700 1 --2- 172.21.15.17:0/407894625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f961410a140 0x7f96140ff510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:45.395 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.393+0000 7f961893c700 1 -- 172.21.15.17:0/407894625 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9614101700 msgr2=0x7f96140fef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:45.395 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.393+0000 7f961893c700 1 --2- 172.21.15.17:0/407894625 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9614101700 0x7f96140fef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:45.395 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.393+0000 7f961893c700 1 -- 172.21.15.17:0/407894625 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9600009520 con 0x7f9614100df0 2023-07-14T12:00:45.396 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.393+0000 7f9613fff700 1 --2- 172.21.15.17:0/407894625 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9614101700 0x7f96140fef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T12:00:45.396 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.394+0000 7f961893c700 1 --2- 172.21.15.17:0/407894625 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9614100df0 0x7f96140fea50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f960800e9a0 tx=0x7f960800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:45.396 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.394+0000 7f9611ffb700 1 -- 172.21.15.17:0/407894625 <== mon.2 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9608009790 con 0x7f9614100df0 2023-07-14T12:00:45.396 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.394+0000 7f961aba0700 1 -- 172.21.15.17:0/407894625 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96141a4790 con 0x7f9614100df0 2023-07-14T12:00:45.396 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.394+0000 7f9611ffb700 1 -- 172.21.15.17:0/407894625 <== mon.2 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9608010430 con 0x7f9614100df0 2023-07-14T12:00:45.396 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.394+0000 7f9611ffb700 1 -- 172.21.15.17:0/407894625 <== mon.2 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9608018680 con 0x7f9614100df0 2023-07-14T12:00:45.396 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.394+0000 7f961aba0700 1 -- 172.21.15.17:0/407894625 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96141a4c60 con 0x7f9614100df0 2023-07-14T12:00:45.397 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.395+0000 7f961aba0700 1 -- 172.21.15.17:0/407894625 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f961404e210 con 0x7f9614100df0 2023-07-14T12:00:45.397 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.397+0000 7f9611ffb700 1 -- 172.21.15.17:0/407894625 <== mon.2 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f96080105a0 con 0x7f9614100df0 2023-07-14T12:00:45.397 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.400+0000 7f9611ffb700 1 --2- 172.21.15.17:0/407894625 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f96040a1150 0x7f96040a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:45.397 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.400+0000 7f9611ffb700 1 -- 172.21.15.17:0/407894625 <== mon.2 v2:172.21.15.93:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9608014070 con 0x7f9614100df0 2023-07-14T12:00:45.397 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.400+0000 7f9613fff700 1 --2- 172.21.15.17:0/407894625 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f96040a1150 0x7f96040a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:45.397 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.400+0000 7f9611ffb700 1 --2- 172.21.15.17:0/407894625 >> [v2:172.21.15.97:6824/3389913754,v1:172.21.15.97:6825/3389913754] conn(0x7f96040ac160 0x7f96040ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:45.397 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.400+0000 7f9611ffb700 1 -- 172.21.15.17:0/407894625 --> [v2:172.21.15.97:6824/3389913754,v1:172.21.15.97:6825/3389913754] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f96040aecc0 con 0x7f96040ac160 2023-07-14T12:00:45.397 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.400+0000 7f9611ffb700 1 -- 172.21.15.17:0/407894625 <== mon.2 v2:172.21.15.93:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96080cdb90 con 0x7f9614100df0 2023-07-14T12:00:45.398 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.401+0000 7f961913d700 1 --2- 172.21.15.17:0/407894625 >> [v2:172.21.15.97:6824/3389913754,v1:172.21.15.97:6825/3389913754] conn(0x7f96040ac160 0x7f96040ae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:45.398 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.401+0000 7f9613fff700 1 --2- 172.21.15.17:0/407894625 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f96040a1150 0x7f96040a3600 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9600005e90 tx=0x7f9600019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:45.398 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.401+0000 7f961913d700 1 --2- 172.21.15.17:0/407894625 >> [v2:172.21.15.97:6824/3389913754,v1:172.21.15.97:6825/3389913754] conn(0x7f96040ac160 0x7f96040ae5f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:45.398 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.402+0000 7f9611ffb700 1 -- 172.21.15.17:0/407894625 <== osd.11 v2:172.21.15.97:6824/3389913754 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f96040aecc0 con 0x7f96040ac160 2023-07-14T12:00:45.401 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.421+0000 7f961aba0700 1 -- 172.21.15.17:0/407894625 --> [v2:172.21.15.97:6824/3389913754,v1:172.21.15.97:6825/3389913754] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f961410fd80 con 0x7f96040ac160 2023-07-14T12:00:45.402 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.422+0000 7f9611ffb700 1 -- 172.21.15.17:0/407894625 <== osd.11 v2:172.21.15.97:6824/3389913754 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f960c00f040 con 0x7f96040ac160 2023-07-14T12:00:45.402 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.422+0000 7f961aba0700 1 -- 172.21.15.17:0/407894625 >> [v2:172.21.15.97:6824/3389913754,v1:172.21.15.97:6825/3389913754] conn(0x7f96040ac160 msgr2=0x7f96040ae5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:45.402 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.422+0000 7f961aba0700 1 --2- 172.21.15.17:0/407894625 >> [v2:172.21.15.97:6824/3389913754,v1:172.21.15.97:6825/3389913754] conn(0x7f96040ac160 0x7f96040ae5f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.422+0000 7f961aba0700 1 -- 172.21.15.17:0/407894625 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f96040a1150 msgr2=0x7f96040a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.422+0000 7f961aba0700 1 --2- 172.21.15.17:0/407894625 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f96040a1150 0x7f96040a3600 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9600005e90 tx=0x7f9600019040 comp rx=0 tx=0).stop 2023-07-14T12:00:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.422+0000 7f961aba0700 1 -- 172.21.15.17:0/407894625 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9614100df0 msgr2=0x7f96140fea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.422+0000 7f961aba0700 1 --2- 172.21.15.17:0/407894625 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9614100df0 0x7f96140fea50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f960800e9a0 tx=0x7f960800ecb0 comp rx=0 tx=0).stop 2023-07-14T12:00:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.422+0000 7f961913d700 1 -- 172.21.15.17:0/407894625 reap_dead start 2023-07-14T12:00:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.422+0000 7f961aba0700 1 -- 172.21.15.17:0/407894625 shutdown_connections 2023-07-14T12:00:45.404 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.422+0000 7f961aba0700 1 -- 172.21.15.17:0/407894625 >> 172.21.15.17:0/407894625 conn(0x7f9614075b70 msgr2=0x7f9614101b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:45.404 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.422+0000 7f961aba0700 1 -- 172.21.15.17:0/407894625 shutdown_connections 2023-07-14T12:00:45.404 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.423+0000 7f961aba0700 1 -- 172.21.15.17:0/407894625 wait complete. 2023-07-14T12:00:45.455 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.474+0000 7f1a7aa53700 1 -- 172.21.15.17:0/1237708233 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1a740731c0 msgr2=0x7f1a7410bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:45.456 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.474+0000 7f1a7aa53700 1 --2- 172.21.15.17:0/1237708233 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1a740731c0 0x7f1a7410bea0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1a68009990 tx=0x7f1a68009ca0 comp rx=0 tx=0).stop 2023-07-14T12:00:45.456 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.475+0000 7f1a7aa53700 1 -- 172.21.15.17:0/1237708233 shutdown_connections 2023-07-14T12:00:45.456 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.475+0000 7f1a7aa53700 1 --2- 172.21.15.17:0/1237708233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a7410c3e0 0x7f1a7410e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:45.456 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.475+0000 7f1a7aa53700 1 --2- 172.21.15.17:0/1237708233 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a740728b0 0x7f1a74072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:45.456 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.475+0000 7f1a7aa53700 1 --2- 172.21.15.17:0/1237708233 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1a740731c0 0x7f1a7410bea0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:45.456 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.475+0000 7f1a7aa53700 1 -- 172.21.15.17:0/1237708233 >> 172.21.15.17:0/1237708233 conn(0x7f1a740fb660 msgr2=0x7f1a740fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:45.456 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.475+0000 7f1a7aa53700 1 -- 172.21.15.17:0/1237708233 shutdown_connections 2023-07-14T12:00:45.457 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.475+0000 7f1a7aa53700 1 -- 172.21.15.17:0/1237708233 wait complete. 2023-07-14T12:00:45.457 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.476+0000 7f1a7aa53700 1 Processor -- start 2023-07-14T12:00:45.458 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.476+0000 7f1a7aa53700 1 -- start start 2023-07-14T12:00:45.458 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.476+0000 7f1a7aa53700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1a740728b0 0x7f1a74197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:45.458 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.476+0000 7f1a7aa53700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a740731c0 0x7f1a74198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:45.458 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.476+0000 7f1a7aa53700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a7410c3e0 0x7f1a7419c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:45.458 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.476+0000 7f1a7aa53700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7419ce90 con 0x7f1a7410c3e0 2023-07-14T12:00:45.458 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.476+0000 7f1a7aa53700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7419d000 con 0x7f1a740728b0 2023-07-14T12:00:45.459 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.476+0000 7f1a7aa53700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7419d1c0 con 0x7f1a740731c0 2023-07-14T12:00:45.459 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.477+0000 7f1a73fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1a740728b0 0x7f1a74197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:45.459 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.477+0000 7f1a73fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1a740728b0 0x7f1a74197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.17:39530/0 (socket says 172.21.15.17:39530) 2023-07-14T12:00:45.459 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.477+0000 7f1a73fff700 1 -- 172.21.15.17:0/3582431400 learned_addr learned my addr 172.21.15.17:0/3582431400 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:00:45.459 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.477+0000 7f1a737fe700 1 --2- 172.21.15.17:0/3582431400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a740731c0 0x7f1a74198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:45.459 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.477+0000 7f1a78ff0700 1 --2- 172.21.15.17:0/3582431400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a7410c3e0 0x7f1a7419c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:45.459 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.477+0000 7f1a737fe700 1 -- 172.21.15.17:0/3582431400 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1a740728b0 msgr2=0x7f1a74197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:45.459 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.477+0000 7f1a737fe700 1 --2- 172.21.15.17:0/3582431400 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1a740728b0 0x7f1a74197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:45.460 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.477+0000 7f1a737fe700 1 -- 172.21.15.17:0/3582431400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a7410c3e0 msgr2=0x7f1a7419c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:45.460 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.477+0000 7f1a737fe700 1 --2- 172.21.15.17:0/3582431400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a7410c3e0 0x7f1a7419c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:45.460 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.477+0000 7f1a737fe700 1 -- 172.21.15.17:0/3582431400 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a680095f0 con 0x7f1a740731c0 2023-07-14T12:00:45.460 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.477+0000 7f1a737fe700 1 --2- 172.21.15.17:0/3582431400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a740731c0 0x7f1a74198380 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1a68009990 tx=0x7f1a68009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:45.461 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.478+0000 7f1a717fa700 1 -- 172.21.15.17:0/3582431400 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6801d070 con 0x7f1a740731c0 2023-07-14T12:00:45.461 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.478+0000 7f1a7aa53700 1 -- 172.21.15.17:0/3582431400 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a7419d440 con 0x7f1a740731c0 2023-07-14T12:00:45.461 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.478+0000 7f1a717fa700 1 -- 172.21.15.17:0/3582431400 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a68021410 con 0x7f1a740731c0 2023-07-14T12:00:45.461 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.478+0000 7f1a717fa700 1 -- 172.21.15.17:0/3582431400 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a68017650 con 0x7f1a740731c0 2023-07-14T12:00:45.461 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.478+0000 7f1a7aa53700 1 -- 172.21.15.17:0/3582431400 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a741a4c70 con 0x7f1a740731c0 2023-07-14T12:00:45.461 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.479+0000 7f1a7aa53700 1 -- 172.21.15.17:0/3582431400 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1a7404e210 con 0x7f1a740731c0 2023-07-14T12:00:45.462 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.481+0000 7f1a717fa700 1 -- 172.21.15.17:0/3582431400 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1a68021810 con 0x7f1a740731c0 2023-07-14T12:00:45.463 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.483+0000 7f1a717fa700 1 --2- 172.21.15.17:0/3582431400 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f1a5c0a1150 0x7f1a5c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:45.463 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.483+0000 7f1a717fa700 1 -- 172.21.15.17:0/3582431400 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1a680cea30 con 0x7f1a740731c0 2023-07-14T12:00:45.463 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.483+0000 7f1a73fff700 1 --2- 172.21.15.17:0/3582431400 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f1a5c0a1150 0x7f1a5c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:45.464 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.483+0000 7f1a717fa700 1 --2- 172.21.15.17:0/3582431400 >> [v2:172.21.15.93:6800/3147203064,v1:172.21.15.93:6801/3147203064] conn(0x7f1a5c0ac160 0x7f1a5c0ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:45.465 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.483+0000 7f1a717fa700 1 -- 172.21.15.17:0/3582431400 --> [v2:172.21.15.93:6800/3147203064,v1:172.21.15.93:6801/3147203064] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1a5c0aecc0 con 0x7f1a5c0ac160 2023-07-14T12:00:45.465 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.483+0000 7f1a717fa700 1 -- 172.21.15.17:0/3582431400 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a680ceda0 con 0x7f1a740731c0 2023-07-14T12:00:45.465 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.483+0000 7f1a78ff0700 1 --2- 172.21.15.17:0/3582431400 >> [v2:172.21.15.93:6800/3147203064,v1:172.21.15.93:6801/3147203064] conn(0x7f1a5c0ac160 0x7f1a5c0ae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:45.465 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.484+0000 7f1a73fff700 1 --2- 172.21.15.17:0/3582431400 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f1a5c0a1150 0x7f1a5c0a3600 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1a64009d20 tx=0x7f1a64009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:45.465 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.484+0000 7f1a78ff0700 1 --2- 172.21.15.17:0/3582431400 >> [v2:172.21.15.93:6800/3147203064,v1:172.21.15.93:6801/3147203064] conn(0x7f1a5c0ac160 0x7f1a5c0ae5f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:45.466 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.485+0000 7f1a717fa700 1 -- 172.21.15.17:0/3582431400 <== osd.1 v2:172.21.15.93:6800/3147203064 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f1a5c0aecc0 con 0x7f1a5c0ac160 2023-07-14T12:00:45.482 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.502+0000 7f1a7aa53700 1 -- 172.21.15.17:0/3582431400 --> [v2:172.21.15.93:6800/3147203064,v1:172.21.15.93:6801/3147203064] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1a74111f80 con 0x7f1a5c0ac160 2023-07-14T12:00:45.482 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.502+0000 7f1a717fa700 1 -- 172.21.15.17:0/3582431400 <== osd.1 v2:172.21.15.93:6800/3147203064 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1a6c00f040 con 0x7f1a5c0ac160 2023-07-14T12:00:45.483 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.503+0000 7f1a7aa53700 1 -- 172.21.15.17:0/3582431400 >> [v2:172.21.15.93:6800/3147203064,v1:172.21.15.93:6801/3147203064] conn(0x7f1a5c0ac160 msgr2=0x7f1a5c0ae5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:45.483 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.503+0000 7f1a7aa53700 1 --2- 172.21.15.17:0/3582431400 >> [v2:172.21.15.93:6800/3147203064,v1:172.21.15.93:6801/3147203064] conn(0x7f1a5c0ac160 0x7f1a5c0ae5f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:45.483 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.503+0000 7f1a7aa53700 1 -- 172.21.15.17:0/3582431400 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f1a5c0a1150 msgr2=0x7f1a5c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:45.484 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.503+0000 7f1a7aa53700 1 --2- 172.21.15.17:0/3582431400 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f1a5c0a1150 0x7f1a5c0a3600 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1a64009d20 tx=0x7f1a64009260 comp rx=0 tx=0).stop 2023-07-14T12:00:45.484 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.503+0000 7f1a7aa53700 1 -- 172.21.15.17:0/3582431400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a740731c0 msgr2=0x7f1a74198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:45.484 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.503+0000 7f1a7aa53700 1 --2- 172.21.15.17:0/3582431400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a740731c0 0x7f1a74198380 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1a68009990 tx=0x7f1a68009580 comp rx=0 tx=0).stop 2023-07-14T12:00:45.484 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.503+0000 7f1a78ff0700 1 -- 172.21.15.17:0/3582431400 reap_dead start 2023-07-14T12:00:45.484 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.503+0000 7f1a7aa53700 1 -- 172.21.15.17:0/3582431400 shutdown_connections 2023-07-14T12:00:45.484 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.503+0000 7f1a7aa53700 1 -- 172.21.15.17:0/3582431400 >> 172.21.15.17:0/3582431400 conn(0x7f1a740fb660 msgr2=0x7f1a7410e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:45.485 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.503+0000 7f1a7aa53700 1 -- 172.21.15.17:0/3582431400 shutdown_connections 2023-07-14T12:00:45.485 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:45.503+0000 7f1a7aa53700 1 -- 172.21.15.17:0/3582431400 wait complete. 2023-07-14T12:00:46.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:45 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T12:00:46.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:45 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T12:00:46.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:45 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T12:00:46.228 INFO:teuthology.orchestra.run.smithi017.stdout:287762808865 2023-07-14T12:00:46.228 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.9 2023-07-14T12:00:46.330 INFO:teuthology.orchestra.run.smithi017.stdout:55834574919 2023-07-14T12:00:46.331 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.1 2023-07-14T12:00:46.917 INFO:teuthology.orchestra.run.smithi017.stdout:339302416408 2023-07-14T12:00:46.918 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.11 2023-07-14T12:00:47.087 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T12:00:47.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:46 smithi093 ceph-mon[128922]: pgmap v310: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2023-07-14T12:00:47.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:46 smithi097 ceph-mon[129313]: pgmap v310: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2023-07-14T12:00:47.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:46 smithi017 ceph-mon[126339]: pgmap v310: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2023-07-14T12:00:47.163 INFO:teuthology.orchestra.run.smithi017.stdout:201863462958 2023-07-14T12:00:47.164 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.6 2023-07-14T12:00:47.204 INFO:teuthology.orchestra.run.smithi017.stdout:309237645340 2023-07-14T12:00:47.204 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.10 2023-07-14T12:00:47.447 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T12:00:47.478 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T12:00:47.482 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T12:00:47.786 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T12:00:47.860 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T12:00:47.869 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T12:00:47.869 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T12:00:49.030 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:48 smithi017 ceph-mon[126339]: pgmap v311: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-07-14T12:00:49.031 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:48 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T12:00:49.031 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:48 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T12:00:49.031 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:48 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T12:00:49.031 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:48 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T12:00:49.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:48 smithi093 ceph-mon[128922]: pgmap v311: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-07-14T12:00:49.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:48 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T12:00:49.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:48 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T12:00:49.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:48 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T12:00:49.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:48 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T12:00:49.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:48 smithi097 ceph-mon[129313]: pgmap v311: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-07-14T12:00:49.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:48 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T12:00:49.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:48 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T12:00:49.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:48 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T12:00:49.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:48 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T12:00:49.287 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T12:00:50.999 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:50 smithi017 ceph-mon[126339]: pgmap v312: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-07-14T12:00:51.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:50 smithi093 ceph-mon[128922]: pgmap v312: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-07-14T12:00:51.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:50 smithi097 ceph-mon[129313]: pgmap v312: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-07-14T12:00:52.150 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.013+0000 7f4e56347700 1 -- 172.21.15.17:0/924192652 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e50101670 msgr2=0x7f4e50109b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:52.150 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.013+0000 7f4e56347700 1 --2- 172.21.15.17:0/924192652 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e50101670 0x7f4e50109b70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4e44009990 tx=0x7f4e44009ca0 comp rx=0 tx=0).stop 2023-07-14T12:00:52.151 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.013+0000 7f4e56347700 1 -- 172.21.15.17:0/924192652 shutdown_connections 2023-07-14T12:00:52.151 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.013+0000 7f4e56347700 1 --2- 172.21.15.17:0/924192652 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4e5010a140 0x7f4e5010c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:52.151 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.013+0000 7f4e56347700 1 --2- 172.21.15.17:0/924192652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e50100d60 0x7f4e50101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:52.151 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.013+0000 7f4e56347700 1 --2- 172.21.15.17:0/924192652 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e50101670 0x7f4e50109b70 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:52.151 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.013+0000 7f4e56347700 1 -- 172.21.15.17:0/924192652 >> 172.21.15.17:0/924192652 conn(0x7f4e500fa5a0 msgr2=0x7f4e500fc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:52.151 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.013+0000 7f4e56347700 1 -- 172.21.15.17:0/924192652 shutdown_connections 2023-07-14T12:00:52.151 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.013+0000 7f4e56347700 1 -- 172.21.15.17:0/924192652 wait complete. 2023-07-14T12:00:52.152 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.014+0000 7f4e56347700 1 Processor -- start 2023-07-14T12:00:52.152 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.015+0000 7f4e56347700 1 -- start start 2023-07-14T12:00:52.152 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.015+0000 7f4e56347700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e50100d60 0x7f4e500fe9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:52.152 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.015+0000 7f4e56347700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e50101670 0x7f4e500fef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:52.152 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.015+0000 7f4e56347700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4e5010a140 0x7f4e500ff480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:52.152 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.015+0000 7f4e56347700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e501a40e0 con 0x7f4e50101670 2023-07-14T12:00:52.152 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.015+0000 7f4e56347700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e501a4250 con 0x7f4e5010a140 2023-07-14T12:00:52.153 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.015+0000 7f4e56347700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e501a4410 con 0x7f4e50100d60 2023-07-14T12:00:52.153 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.015+0000 7f4e4f7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e50101670 0x7f4e500fef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:52.153 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.015+0000 7f4e4f7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e50101670 0x7f4e500fef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37982/0 (socket says 172.21.15.17:37982) 2023-07-14T12:00:52.153 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.015+0000 7f4e4ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e50100d60 0x7f4e500fe9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:52.153 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.015+0000 7f4e4f7fe700 1 -- 172.21.15.17:0/2305391509 learned_addr learned my addr 172.21.15.17:0/2305391509 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:00:52.153 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.015+0000 7f4e548e4700 1 --2- 172.21.15.17:0/2305391509 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4e5010a140 0x7f4e500ff480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:52.153 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.016+0000 7f4e4f7fe700 1 -- 172.21.15.17:0/2305391509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e50100d60 msgr2=0x7f4e500fe9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:52.153 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.016+0000 7f4e4f7fe700 1 --2- 172.21.15.17:0/2305391509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e50100d60 0x7f4e500fe9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:52.154 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.016+0000 7f4e4f7fe700 1 -- 172.21.15.17:0/2305391509 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4e5010a140 msgr2=0x7f4e500ff480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:52.154 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.016+0000 7f4e4f7fe700 1 --2- 172.21.15.17:0/2305391509 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4e5010a140 0x7f4e500ff480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:52.154 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.016+0000 7f4e4f7fe700 1 -- 172.21.15.17:0/2305391509 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e440095f0 con 0x7f4e50101670 2023-07-14T12:00:52.154 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.016+0000 7f4e4f7fe700 1 --2- 172.21.15.17:0/2305391509 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e50101670 0x7f4e500fef00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4e44005910 tx=0x7f4e44005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:52.154 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.016+0000 7f4e4d7fa700 1 -- 172.21.15.17:0/2305391509 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4401d070 con 0x7f4e50101670 2023-07-14T12:00:52.154 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.016+0000 7f4e56347700 1 -- 172.21.15.17:0/2305391509 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e501a4690 con 0x7f4e50101670 2023-07-14T12:00:52.154 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.016+0000 7f4e4d7fa700 1 -- 172.21.15.17:0/2305391509 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e44022470 con 0x7f4e50101670 2023-07-14T12:00:52.155 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.016+0000 7f4e4d7fa700 1 -- 172.21.15.17:0/2305391509 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e44017650 con 0x7f4e50101670 2023-07-14T12:00:52.155 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.017+0000 7f4e56347700 1 -- 172.21.15.17:0/2305391509 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e501a4b30 con 0x7f4e50101670 2023-07-14T12:00:52.155 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.018+0000 7f4e56347700 1 -- 172.21.15.17:0/2305391509 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e500681a0 con 0x7f4e50101670 2023-07-14T12:00:52.155 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.020+0000 7f4e4d7fa700 1 -- 172.21.15.17:0/2305391509 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4e4400fad0 con 0x7f4e50101670 2023-07-14T12:00:52.155 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.022+0000 7f4e4d7fa700 1 --2- 172.21.15.17:0/2305391509 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f4e380a1150 0x7f4e380a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:52.155 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.023+0000 7f4e4d7fa700 1 -- 172.21.15.17:0/2305391509 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4e440ce830 con 0x7f4e50101670 2023-07-14T12:00:52.155 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.023+0000 7f4e4ffff700 1 --2- 172.21.15.17:0/2305391509 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f4e380a1150 0x7f4e380a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:52.155 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.024+0000 7f4e4ffff700 1 --2- 172.21.15.17:0/2305391509 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f4e380a1150 0x7f4e380a3600 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4e40007be0 tx=0x7f4e400073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:52.156 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.024+0000 7f4e4d7fa700 1 -- 172.21.15.17:0/2305391509 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4e44097ca0 con 0x7f4e50101670 2023-07-14T12:00:52.228 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.248+0000 7f4e56347700 1 -- 172.21.15.17:0/2305391509 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4e5004e210 con 0x7f4e50101670 2023-07-14T12:00:52.229 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.248+0000 7f4e4d7fa700 1 -- 172.21.15.17:0/2305391509 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4e4409b2c0 con 0x7f4e50101670 2023-07-14T12:00:52.229 INFO:teuthology.orchestra.run.smithi017.stdout:111669149758 2023-07-14T12:00:52.231 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.251+0000 7f4e56347700 1 -- 172.21.15.17:0/2305391509 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f4e380a1150 msgr2=0x7f4e380a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:52.231 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.251+0000 7f4e56347700 1 --2- 172.21.15.17:0/2305391509 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f4e380a1150 0x7f4e380a3600 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4e40007be0 tx=0x7f4e400073d0 comp rx=0 tx=0).stop 2023-07-14T12:00:52.231 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.251+0000 7f4e56347700 1 -- 172.21.15.17:0/2305391509 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e50101670 msgr2=0x7f4e500fef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:52.231 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.251+0000 7f4e56347700 1 --2- 172.21.15.17:0/2305391509 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e50101670 0x7f4e500fef00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4e44005910 tx=0x7f4e44005740 comp rx=0 tx=0).stop 2023-07-14T12:00:52.232 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.251+0000 7f4e56347700 1 -- 172.21.15.17:0/2305391509 shutdown_connections 2023-07-14T12:00:52.232 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.251+0000 7f4e56347700 1 --2- 172.21.15.17:0/2305391509 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f4e380a1150 0x7f4e380a3600 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:52.232 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.251+0000 7f4e56347700 1 --2- 172.21.15.17:0/2305391509 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4e5010a140 0x7f4e500ff480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:52.232 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.251+0000 7f4e56347700 1 --2- 172.21.15.17:0/2305391509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e50100d60 0x7f4e500fe9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:52.232 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.251+0000 7f4e56347700 1 --2- 172.21.15.17:0/2305391509 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e50101670 0x7f4e500fef00 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:52.233 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.251+0000 7f4e56347700 1 -- 172.21.15.17:0/2305391509 >> 172.21.15.17:0/2305391509 conn(0x7f4e500fa5a0 msgr2=0x7f4e5010c330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:52.233 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.251+0000 7f4e56347700 1 -- 172.21.15.17:0/2305391509 shutdown_connections 2023-07-14T12:00:52.233 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.251+0000 7f4e56347700 1 -- 172.21.15.17:0/2305391509 wait complete. 2023-07-14T12:00:52.666 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.587+0000 7fa2aeb72700 1 -- 172.21.15.17:0/2095745593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2a8101700 msgr2=0x7fa2a8109c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:52.666 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.587+0000 7fa2aeb72700 1 --2- 172.21.15.17:0/2095745593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2a8101700 0x7fa2a8109c00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa2a4009f90 tx=0x7fa2a400b330 comp rx=0 tx=0).stop 2023-07-14T12:00:52.666 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.587+0000 7fa2aeb72700 1 -- 172.21.15.17:0/2095745593 shutdown_connections 2023-07-14T12:00:52.666 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.587+0000 7fa2aeb72700 1 --2- 172.21.15.17:0/2095745593 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2a810a140 0x7fa2a810c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:52.667 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.587+0000 7fa2aeb72700 1 --2- 172.21.15.17:0/2095745593 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2a8100df0 0x7fa2a81011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:52.667 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.587+0000 7fa2aeb72700 1 --2- 172.21.15.17:0/2095745593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2a8101700 0x7fa2a8109c00 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:52.667 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.587+0000 7fa2aeb72700 1 -- 172.21.15.17:0/2095745593 >> 172.21.15.17:0/2095745593 conn(0x7fa2a8075b70 msgr2=0x7fa2a8075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:52.667 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.587+0000 7fa2aeb72700 1 -- 172.21.15.17:0/2095745593 shutdown_connections 2023-07-14T12:00:52.667 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.587+0000 7fa2aeb72700 1 -- 172.21.15.17:0/2095745593 wait complete. 2023-07-14T12:00:52.667 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.589+0000 7fa2aeb72700 1 Processor -- start 2023-07-14T12:00:52.667 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.590+0000 7fa2aeb72700 1 -- start start 2023-07-14T12:00:52.667 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.590+0000 7fa2aeb72700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2a8100df0 0x7fa2a81097e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:52.668 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.590+0000 7fa2aeb72700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2a8101700 0x7fa2a8107860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:52.668 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.590+0000 7fa2aeb72700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2a810a140 0x7fa2a8107da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:52.668 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.590+0000 7fa2aeb72700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a81083a0 con 0x7fa2a8100df0 2023-07-14T12:00:52.668 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.590+0000 7fa2aeb72700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a8108510 con 0x7fa2a8101700 2023-07-14T12:00:52.668 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.590+0000 7fa2aeb72700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a8108680 con 0x7fa2a810a140 2023-07-14T12:00:52.668 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.591+0000 7fa29ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2a8101700 0x7fa2a8107860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:52.668 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.591+0000 7fa29ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2a8101700 0x7fa2a8107860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.17:39550/0 (socket says 172.21.15.17:39550) 2023-07-14T12:00:52.669 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.591+0000 7fa29ffff700 1 -- 172.21.15.17:0/3097953571 learned_addr learned my addr 172.21.15.17:0/3097953571 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:00:52.669 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.591+0000 7fa2ac90e700 1 --2- 172.21.15.17:0/3097953571 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2a8100df0 0x7fa2a81097e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:52.669 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.591+0000 7fa2ad10f700 1 --2- 172.21.15.17:0/3097953571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2a810a140 0x7fa2a8107da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:52.669 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.591+0000 7fa29ffff700 1 -- 172.21.15.17:0/3097953571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2a810a140 msgr2=0x7fa2a8107da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:52.669 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.591+0000 7fa29ffff700 1 --2- 172.21.15.17:0/3097953571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2a810a140 0x7fa2a8107da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:52.669 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.591+0000 7fa29ffff700 1 -- 172.21.15.17:0/3097953571 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2a8100df0 msgr2=0x7fa2a81097e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:52.669 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.591+0000 7fa29ffff700 1 --2- 172.21.15.17:0/3097953571 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2a8100df0 0x7fa2a81097e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:52.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.591+0000 7fa29ffff700 1 -- 172.21.15.17:0/3097953571 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a4009c40 con 0x7fa2a8101700 2023-07-14T12:00:52.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.592+0000 7fa29ffff700 1 --2- 172.21.15.17:0/3097953571 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2a8101700 0x7fa2a8107860 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400b6c0 tx=0x7fa2a4004ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:52.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.592+0000 7fa29dffb700 1 -- 172.21.15.17:0/3097953571 <== mon.2 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a400e040 con 0x7fa2a8101700 2023-07-14T12:00:52.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.592+0000 7fa2aeb72700 1 -- 172.21.15.17:0/3097953571 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2a8119710 con 0x7fa2a8101700 2023-07-14T12:00:52.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.592+0000 7fa2aeb72700 1 -- 172.21.15.17:0/3097953571 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2a8119c30 con 0x7fa2a8101700 2023-07-14T12:00:52.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.593+0000 7fa29dffb700 1 -- 172.21.15.17:0/3097953571 <== mon.2 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2a4008ec0 con 0x7fa2a8101700 2023-07-14T12:00:52.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.593+0000 7fa29dffb700 1 -- 172.21.15.17:0/3097953571 <== mon.2 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a4005330 con 0x7fa2a8101700 2023-07-14T12:00:52.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.595+0000 7fa2aeb72700 1 -- 172.21.15.17:0/3097953571 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2a804e210 con 0x7fa2a8101700 2023-07-14T12:00:52.671 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.597+0000 7fa29dffb700 1 -- 172.21.15.17:0/3097953571 <== mon.2 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa2a401d470 con 0x7fa2a8101700 2023-07-14T12:00:52.671 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.599+0000 7fa29dffb700 1 --2- 172.21.15.17:0/3097953571 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fa2900a1250 0x7fa2900a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:52.671 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.599+0000 7fa29dffb700 1 -- 172.21.15.17:0/3097953571 <== mon.2 v2:172.21.15.93:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa2a40ce100 con 0x7fa2a8101700 2023-07-14T12:00:52.671 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.599+0000 7fa2ac90e700 1 --2- 172.21.15.17:0/3097953571 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fa2900a1250 0x7fa2900a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:52.671 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.600+0000 7fa2ac90e700 1 --2- 172.21.15.17:0/3097953571 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fa2900a1250 0x7fa2900a3700 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa298007be0 tx=0x7fa2980073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:52.671 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.602+0000 7fa29dffb700 1 -- 172.21.15.17:0/3097953571 <== mon.2 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa2a4097570 con 0x7fa2a8101700 2023-07-14T12:00:52.803 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.823+0000 7fa2aeb72700 1 -- 172.21.15.17:0/3097953571 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7fa2a8119f60 con 0x7fa2a8101700 2023-07-14T12:00:52.804 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.823+0000 7fa29dffb700 1 -- 172.21.15.17:0/3097953571 <== mon.2 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa2a409ab90 con 0x7fa2a8101700 2023-07-14T12:00:52.804 INFO:teuthology.orchestra.run.smithi017.stdout:253403070504 2023-07-14T12:00:52.806 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.826+0000 7fa2aeb72700 1 -- 172.21.15.17:0/3097953571 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fa2900a1250 msgr2=0x7fa2900a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:52.806 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.826+0000 7fa2aeb72700 1 --2- 172.21.15.17:0/3097953571 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fa2900a1250 0x7fa2900a3700 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa298007be0 tx=0x7fa2980073d0 comp rx=0 tx=0).stop 2023-07-14T12:00:52.807 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.826+0000 7fa2aeb72700 1 -- 172.21.15.17:0/3097953571 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2a8101700 msgr2=0x7fa2a8107860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:52.807 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.826+0000 7fa2aeb72700 1 --2- 172.21.15.17:0/3097953571 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2a8101700 0x7fa2a8107860 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400b6c0 tx=0x7fa2a4004ce0 comp rx=0 tx=0).stop 2023-07-14T12:00:52.807 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.826+0000 7fa2aeb72700 1 -- 172.21.15.17:0/3097953571 shutdown_connections 2023-07-14T12:00:52.807 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.826+0000 7fa2aeb72700 1 --2- 172.21.15.17:0/3097953571 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fa2900a1250 0x7fa2900a3700 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:52.807 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.826+0000 7fa2aeb72700 1 --2- 172.21.15.17:0/3097953571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2a810a140 0x7fa2a8107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:52.807 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.826+0000 7fa2aeb72700 1 --2- 172.21.15.17:0/3097953571 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2a8100df0 0x7fa2a81097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:52.807 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.826+0000 7fa2aeb72700 1 --2- 172.21.15.17:0/3097953571 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2a8101700 0x7fa2a8107860 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:52.807 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.826+0000 7fa2aeb72700 1 -- 172.21.15.17:0/3097953571 >> 172.21.15.17:0/3097953571 conn(0x7fa2a8075b70 msgr2=0x7fa2a810e530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:52.808 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.826+0000 7fa2aeb72700 1 -- 172.21.15.17:0/3097953571 shutdown_connections 2023-07-14T12:00:52.808 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.826+0000 7fa2aeb72700 1 -- 172.21.15.17:0/3097953571 wait complete. 2023-07-14T12:00:52.886 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.905+0000 7f046a7f5700 1 -- 172.21.15.17:0/1950921606 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04640728b0 msgr2=0x7f0464072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:52.886 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.905+0000 7f046a7f5700 1 --2- 172.21.15.17:0/1950921606 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04640728b0 0x7f0464072c80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0454009990 tx=0x7f0454009ca0 comp rx=0 tx=0).stop 2023-07-14T12:00:52.886 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.906+0000 7f046a7f5700 1 -- 172.21.15.17:0/1950921606 shutdown_connections 2023-07-14T12:00:52.886 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.906+0000 7f046a7f5700 1 --2- 172.21.15.17:0/1950921606 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0464110bb0 0x7f0464112f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:52.887 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.906+0000 7f046a7f5700 1 --2- 172.21.15.17:0/1950921606 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04640728b0 0x7f0464072c80 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:52.887 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.906+0000 7f046a7f5700 1 --2- 172.21.15.17:0/1950921606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04640731c0 0x7f0464110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:52.887 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.906+0000 7f046a7f5700 1 -- 172.21.15.17:0/1950921606 >> 172.21.15.17:0/1950921606 conn(0x7f04640fb660 msgr2=0x7f04640fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:52.887 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.906+0000 7f046a7f5700 1 -- 172.21.15.17:0/1950921606 shutdown_connections 2023-07-14T12:00:52.887 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.906+0000 7f046a7f5700 1 -- 172.21.15.17:0/1950921606 wait complete. 2023-07-14T12:00:52.888 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.907+0000 7f046a7f5700 1 Processor -- start 2023-07-14T12:00:52.888 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.907+0000 7f046a7f5700 1 -- start start 2023-07-14T12:00:52.888 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.907+0000 7f046a7f5700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f04640728b0 0x7f046419c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:52.888 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.907+0000 7f046a7f5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04640731c0 0x7f046419c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:52.888 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.908+0000 7f046a7f5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0464110bb0 0x7f04641a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:52.889 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.908+0000 7f046a7f5700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04641a12d0 con 0x7f0464110bb0 2023-07-14T12:00:52.889 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.908+0000 7f046a7f5700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04641a1440 con 0x7f04640728b0 2023-07-14T12:00:52.889 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.908+0000 7f046a7f5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04641a1600 con 0x7f04640731c0 2023-07-14T12:00:52.889 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.908+0000 7f0468d92700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0464110bb0 0x7f04641a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:52.889 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.908+0000 7f0468d92700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0464110bb0 0x7f04641a0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54974/0 (socket says 172.21.15.17:54974) 2023-07-14T12:00:52.890 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.908+0000 7f0468d92700 1 -- 172.21.15.17:0/796282623 learned_addr learned my addr 172.21.15.17:0/796282623 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:00:52.890 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.908+0000 7f04637fe700 1 --2- 172.21.15.17:0/796282623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04640731c0 0x7f046419c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:52.890 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.908+0000 7f0463fff700 1 --2- 172.21.15.17:0/796282623 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f04640728b0 0x7f046419c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:52.891 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.908+0000 7f0468d92700 1 -- 172.21.15.17:0/796282623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04640731c0 msgr2=0x7f046419c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:52.891 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.908+0000 7f0468d92700 1 --2- 172.21.15.17:0/796282623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04640731c0 0x7f046419c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:52.891 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.908+0000 7f0468d92700 1 -- 172.21.15.17:0/796282623 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f04640728b0 msgr2=0x7f046419c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:52.891 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.908+0000 7f0468d92700 1 --2- 172.21.15.17:0/796282623 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f04640728b0 0x7f046419c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:52.891 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.908+0000 7f0468d92700 1 -- 172.21.15.17:0/796282623 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f045c009c40 con 0x7f0464110bb0 2023-07-14T12:00:52.891 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.908+0000 7f04637fe700 1 --2- 172.21.15.17:0/796282623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04640731c0 0x7f046419c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T12:00:52.891 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.908+0000 7f0463fff700 1 --2- 172.21.15.17:0/796282623 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f04640728b0 0x7f046419c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T12:00:52.892 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.909+0000 7f0468d92700 1 --2- 172.21.15.17:0/796282623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0464110bb0 0x7f04641a0cc0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f045c00c110 tx=0x7f045c00c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:52.892 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.909+0000 7f04617fa700 1 -- 172.21.15.17:0/796282623 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045c017070 con 0x7f0464110bb0 2023-07-14T12:00:52.892 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.909+0000 7f04617fa700 1 -- 172.21.15.17:0/796282623 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f045c00f040 con 0x7f0464110bb0 2023-07-14T12:00:52.892 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.909+0000 7f04617fa700 1 -- 172.21.15.17:0/796282623 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045c00e050 con 0x7f0464110bb0 2023-07-14T12:00:52.892 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.909+0000 7f046a7f5700 1 -- 172.21.15.17:0/796282623 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04540095f0 con 0x7f0464110bb0 2023-07-14T12:00:52.892 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.909+0000 7f046a7f5700 1 -- 172.21.15.17:0/796282623 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04641a8f70 con 0x7f0464110bb0 2023-07-14T12:00:52.893 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.910+0000 7f046a7f5700 1 -- 172.21.15.17:0/796282623 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0464103090 con 0x7f0464110bb0 2023-07-14T12:00:52.894 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.914+0000 7f04617fa700 1 -- 172.21.15.17:0/796282623 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f045c00ca20 con 0x7f0464110bb0 2023-07-14T12:00:52.895 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.915+0000 7f04617fa700 1 --2- 172.21.15.17:0/796282623 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f044c0a1150 0x7f044c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:52.896 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.916+0000 7f04617fa700 1 -- 172.21.15.17:0/796282623 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f045c0cc1f0 con 0x7f0464110bb0 2023-07-14T12:00:52.896 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.916+0000 7f0463fff700 1 --2- 172.21.15.17:0/796282623 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f044c0a1150 0x7f044c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:52.897 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.917+0000 7f0463fff700 1 --2- 172.21.15.17:0/796282623 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f044c0a1150 0x7f044c0a3600 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f045400b2e0 tx=0x7f0454005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:52.901 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:52.921+0000 7f04617fa700 1 -- 172.21.15.17:0/796282623 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f045c095630 con 0x7f0464110bb0 2023-07-14T12:00:53.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:52 smithi093 ceph-mon[128922]: pgmap v313: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-07-14T12:00:53.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:52 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/2305391509' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-07-14T12:00:53.120 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.140+0000 7f046a7f5700 1 -- 172.21.15.17:0/796282623 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f046404e210 con 0x7f0464110bb0 2023-07-14T12:00:53.121 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.140+0000 7f04617fa700 1 -- 172.21.15.17:0/796282623 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f045c098c50 con 0x7f0464110bb0 2023-07-14T12:00:53.121 INFO:teuthology.orchestra.run.smithi017.stdout:77309411395 2023-07-14T12:00:53.123 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.143+0000 7f046a7f5700 1 -- 172.21.15.17:0/796282623 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f044c0a1150 msgr2=0x7f044c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:53.124 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.143+0000 7f046a7f5700 1 --2- 172.21.15.17:0/796282623 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f044c0a1150 0x7f044c0a3600 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f045400b2e0 tx=0x7f0454005fd0 comp rx=0 tx=0).stop 2023-07-14T12:00:53.124 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.143+0000 7f046a7f5700 1 -- 172.21.15.17:0/796282623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0464110bb0 msgr2=0x7f04641a0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:53.124 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.143+0000 7f046a7f5700 1 --2- 172.21.15.17:0/796282623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0464110bb0 0x7f04641a0cc0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f045c00c110 tx=0x7f045c00c420 comp rx=0 tx=0).stop 2023-07-14T12:00:53.124 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.143+0000 7f046a7f5700 1 -- 172.21.15.17:0/796282623 shutdown_connections 2023-07-14T12:00:53.124 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.143+0000 7f046a7f5700 1 --2- 172.21.15.17:0/796282623 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f044c0a1150 0x7f044c0a3600 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:53.124 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.143+0000 7f046a7f5700 1 --2- 172.21.15.17:0/796282623 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f04640728b0 0x7f046419c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:53.124 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.143+0000 7f046a7f5700 1 --2- 172.21.15.17:0/796282623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0464110bb0 0x7f04641a0cc0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:53.125 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.143+0000 7f046a7f5700 1 --2- 172.21.15.17:0/796282623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04640731c0 0x7f046419c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:53.125 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.143+0000 7f046a7f5700 1 -- 172.21.15.17:0/796282623 >> 172.21.15.17:0/796282623 conn(0x7f04640fb660 msgr2=0x7f04640fc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:53.125 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.143+0000 7f046a7f5700 1 -- 172.21.15.17:0/796282623 shutdown_connections 2023-07-14T12:00:53.125 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.143+0000 7f046a7f5700 1 -- 172.21.15.17:0/796282623 wait complete. 2023-07-14T12:00:53.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:52 smithi097 ceph-mon[129313]: pgmap v313: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-07-14T12:00:53.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:52 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/2305391509' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-07-14T12:00:53.143 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:52 smithi017 ceph-mon[126339]: pgmap v313: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-07-14T12:00:53.143 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:52 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/2305391509' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-07-14T12:00:53.950 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.769+0000 7f577612e700 1 -- 172.21.15.17:0/517300092 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57700ffe00 msgr2=0x7f57701001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:53.951 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.769+0000 7f577612e700 1 --2- 172.21.15.17:0/517300092 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57700ffe00 0x7f57701001d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f57600099c0 tx=0x7f5760009cd0 comp rx=0 tx=0).stop 2023-07-14T12:00:53.951 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.769+0000 7f577612e700 1 -- 172.21.15.17:0/517300092 shutdown_connections 2023-07-14T12:00:53.951 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.769+0000 7f577612e700 1 --2- 172.21.15.17:0/517300092 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5770104ea0 0x7f577010ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:53.951 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.769+0000 7f577612e700 1 --2- 172.21.15.17:0/517300092 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57700ffe00 0x7f57701001d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:53.951 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.769+0000 7f577612e700 1 --2- 172.21.15.17:0/517300092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57701007a0 0x7f5770104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:53.951 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.769+0000 7f577612e700 1 -- 172.21.15.17:0/517300092 >> 172.21.15.17:0/517300092 conn(0x7f57700fb660 msgr2=0x7f57700fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:53.952 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.769+0000 7f577612e700 1 -- 172.21.15.17:0/517300092 shutdown_connections 2023-07-14T12:00:53.952 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.769+0000 7f577612e700 1 -- 172.21.15.17:0/517300092 wait complete. 2023-07-14T12:00:53.952 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.770+0000 7f577612e700 1 Processor -- start 2023-07-14T12:00:53.952 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.770+0000 7f577612e700 1 -- start start 2023-07-14T12:00:53.952 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.771+0000 7f577612e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57700ffe00 0x7f5770195cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:53.952 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.771+0000 7f577612e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57701007a0 0x7f57701961f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:53.952 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.771+0000 7f577612e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5770104ea0 0x7f57701995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:53.953 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.771+0000 7f577612e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5770199bb0 con 0x7f5770104ea0 2023-07-14T12:00:53.953 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.771+0000 7f577612e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5770199d20 con 0x7f57701007a0 2023-07-14T12:00:53.953 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.771+0000 7f577612e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5770199ee0 con 0x7f57700ffe00 2023-07-14T12:00:53.953 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.771+0000 7f576ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5770104ea0 0x7f57701995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:53.953 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.771+0000 7f576ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5770104ea0 0x7f57701995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54990/0 (socket says 172.21.15.17:54990) 2023-07-14T12:00:53.953 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.771+0000 7f576ffff700 1 -- 172.21.15.17:0/281593326 learned_addr learned my addr 172.21.15.17:0/281593326 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:00:53.953 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.771+0000 7f576f7fe700 1 --2- 172.21.15.17:0/281593326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57700ffe00 0x7f5770195cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:53.954 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.771+0000 7f576effd700 1 --2- 172.21.15.17:0/281593326 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57701007a0 0x7f57701961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:53.954 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.772+0000 7f576ffff700 1 -- 172.21.15.17:0/281593326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57700ffe00 msgr2=0x7f5770195cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:53.954 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.772+0000 7f576ffff700 1 --2- 172.21.15.17:0/281593326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57700ffe00 0x7f5770195cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:53.954 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.772+0000 7f576ffff700 1 -- 172.21.15.17:0/281593326 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57701007a0 msgr2=0x7f57701961f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:53.954 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.772+0000 7f576ffff700 1 --2- 172.21.15.17:0/281593326 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57701007a0 0x7f57701961f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:53.954 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.772+0000 7f576ffff700 1 -- 172.21.15.17:0/281593326 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57600095f0 con 0x7f5770104ea0 2023-07-14T12:00:53.954 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.772+0000 7f576f7fe700 1 --2- 172.21.15.17:0/281593326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57700ffe00 0x7f5770195cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T12:00:53.954 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.772+0000 7f576ffff700 1 --2- 172.21.15.17:0/281593326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5770104ea0 0x7f57701995b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f576800ef30 tx=0x7f576800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:53.955 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.772+0000 7f576cff9700 1 -- 172.21.15.17:0/281593326 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5768018070 con 0x7f5770104ea0 2023-07-14T12:00:53.955 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.772+0000 7f577612e700 1 -- 172.21.15.17:0/281593326 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f577019a160 con 0x7f5770104ea0 2023-07-14T12:00:53.955 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.772+0000 7f576cff9700 1 -- 172.21.15.17:0/281593326 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f576800f040 con 0x7f5770104ea0 2023-07-14T12:00:53.955 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.772+0000 7f576cff9700 1 -- 172.21.15.17:0/281593326 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576800cd00 con 0x7f5770104ea0 2023-07-14T12:00:53.955 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.772+0000 7f577612e700 1 -- 172.21.15.17:0/281593326 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57701a2b10 con 0x7f5770104ea0 2023-07-14T12:00:53.955 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.774+0000 7f577612e700 1 -- 172.21.15.17:0/281593326 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f577004e210 con 0x7f5770104ea0 2023-07-14T12:00:53.955 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.776+0000 7f576cff9700 1 -- 172.21.15.17:0/281593326 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5768003a00 con 0x7f5770104ea0 2023-07-14T12:00:53.955 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.778+0000 7f576cff9700 1 --2- 172.21.15.17:0/281593326 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f57580a1150 0x7f57580a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:53.956 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.778+0000 7f576cff9700 1 -- 172.21.15.17:0/281593326 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f57680cc6d0 con 0x7f5770104ea0 2023-07-14T12:00:53.956 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.778+0000 7f576f7fe700 1 --2- 172.21.15.17:0/281593326 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f57580a1150 0x7f57580a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:53.956 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.779+0000 7f576f7fe700 1 --2- 172.21.15.17:0/281593326 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f57580a1150 0x7f57580a3600 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5760009990 tx=0x7f5760005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:53.956 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:53.782+0000 7f576cff9700 1 -- 172.21.15.17:0/281593326 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5768095ba0 con 0x7f5770104ea0 2023-07-14T12:00:53.981 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.001+0000 7f577612e700 1 -- 172.21.15.17:0/281593326 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f57701a2d40 con 0x7f5770104ea0 2023-07-14T12:00:53.982 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.002+0000 7f576cff9700 1 -- 172.21.15.17:0/281593326 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f57680991c0 con 0x7f5770104ea0 2023-07-14T12:00:53.982 INFO:teuthology.orchestra.run.smithi017.stdout:287762808866 2023-07-14T12:00:53.985 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.004+0000 7f577612e700 1 -- 172.21.15.17:0/281593326 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f57580a1150 msgr2=0x7f57580a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:53.985 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.004+0000 7f577612e700 1 --2- 172.21.15.17:0/281593326 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f57580a1150 0x7f57580a3600 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5760009990 tx=0x7f5760005fb0 comp rx=0 tx=0).stop 2023-07-14T12:00:53.985 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.004+0000 7f577612e700 1 -- 172.21.15.17:0/281593326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5770104ea0 msgr2=0x7f57701995b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:53.985 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.004+0000 7f577612e700 1 --2- 172.21.15.17:0/281593326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5770104ea0 0x7f57701995b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f576800ef30 tx=0x7f576800c3a0 comp rx=0 tx=0).stop 2023-07-14T12:00:53.985 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.004+0000 7f577612e700 1 -- 172.21.15.17:0/281593326 shutdown_connections 2023-07-14T12:00:53.985 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.004+0000 7f577612e700 1 --2- 172.21.15.17:0/281593326 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f57580a1150 0x7f57580a3600 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:53.985 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.004+0000 7f577612e700 1 --2- 172.21.15.17:0/281593326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5770104ea0 0x7f57701995b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:53.986 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.004+0000 7f577612e700 1 --2- 172.21.15.17:0/281593326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57700ffe00 0x7f5770195cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:53.986 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.004+0000 7f577612e700 1 --2- 172.21.15.17:0/281593326 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57701007a0 0x7f57701961f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:53.986 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.004+0000 7f577612e700 1 -- 172.21.15.17:0/281593326 >> 172.21.15.17:0/281593326 conn(0x7f57700fb660 msgr2=0x7f57700fc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:53.986 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.004+0000 7f577612e700 1 -- 172.21.15.17:0/281593326 shutdown_connections 2023-07-14T12:00:53.986 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.005+0000 7f577612e700 1 -- 172.21.15.17:0/281593326 wait complete. 2023-07-14T12:00:54.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:53 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/3097953571' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-07-14T12:00:54.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:53 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/796282623' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-07-14T12:00:54.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:53 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/3097953571' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-07-14T12:00:54.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:53 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/796282623' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-07-14T12:00:54.143 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:53 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/3097953571' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-07-14T12:00:54.143 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:53 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/796282623' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-07-14T12:00:54.274 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.213+0000 7fafc7f8f700 1 -- 172.21.15.17:0/2379423173 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafc00728b0 msgr2=0x7fafc0072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:54.274 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.213+0000 7fafc7f8f700 1 --2- 172.21.15.17:0/2379423173 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafc00728b0 0x7fafc0072c80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fafbc009990 tx=0x7fafbc009ca0 comp rx=0 tx=0).stop 2023-07-14T12:00:54.274 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.213+0000 7fafc7f8f700 1 -- 172.21.15.17:0/2379423173 shutdown_connections 2023-07-14T12:00:54.274 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.213+0000 7fafc7f8f700 1 --2- 172.21.15.17:0/2379423173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafc010c3e0 0x7fafc010e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:54.275 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.213+0000 7fafc7f8f700 1 --2- 172.21.15.17:0/2379423173 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafc00728b0 0x7fafc0072c80 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:54.275 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.213+0000 7fafc7f8f700 1 --2- 172.21.15.17:0/2379423173 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fafc00731c0 0x7fafc010bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:54.275 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.213+0000 7fafc7f8f700 1 -- 172.21.15.17:0/2379423173 >> 172.21.15.17:0/2379423173 conn(0x7fafc00fb660 msgr2=0x7fafc00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:54.275 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.213+0000 7fafc7f8f700 1 -- 172.21.15.17:0/2379423173 shutdown_connections 2023-07-14T12:00:54.275 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.213+0000 7fafc7f8f700 1 -- 172.21.15.17:0/2379423173 wait complete. 2023-07-14T12:00:54.275 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.214+0000 7fafc7f8f700 1 Processor -- start 2023-07-14T12:00:54.275 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.214+0000 7fafc7f8f700 1 -- start start 2023-07-14T12:00:54.276 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.215+0000 7fafc7f8f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fafc00728b0 0x7fafc0197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:54.276 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.215+0000 7fafc7f8f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafc00731c0 0x7fafc0198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:54.276 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.215+0000 7fafc7f8f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafc010c3e0 0x7fafc019c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:54.276 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.215+0000 7fafc7f8f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc019ce60 con 0x7fafc010c3e0 2023-07-14T12:00:54.276 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.215+0000 7fafc7f8f700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc019cfd0 con 0x7fafc00728b0 2023-07-14T12:00:54.276 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.215+0000 7fafc7f8f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc019d190 con 0x7fafc00731c0 2023-07-14T12:00:54.276 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.215+0000 7fafc652c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafc010c3e0 0x7fafc019c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:54.276 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.215+0000 7fafc652c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafc010c3e0 0x7fafc019c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55016/0 (socket says 172.21.15.17:55016) 2023-07-14T12:00:54.277 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.215+0000 7fafc652c700 1 -- 172.21.15.17:0/399508465 learned_addr learned my addr 172.21.15.17:0/399508465 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:00:54.277 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.215+0000 7fafc5d2b700 1 --2- 172.21.15.17:0/399508465 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fafc00728b0 0x7fafc0197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:54.277 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.215+0000 7fafc552a700 1 --2- 172.21.15.17:0/399508465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafc00731c0 0x7fafc0198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:54.277 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.215+0000 7fafc652c700 1 -- 172.21.15.17:0/399508465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafc00731c0 msgr2=0x7fafc0198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:54.277 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.215+0000 7fafc652c700 1 --2- 172.21.15.17:0/399508465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafc00731c0 0x7fafc0198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:54.277 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.215+0000 7fafc652c700 1 -- 172.21.15.17:0/399508465 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fafc00728b0 msgr2=0x7fafc0197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:54.277 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.215+0000 7fafc652c700 1 --2- 172.21.15.17:0/399508465 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fafc00728b0 0x7fafc0197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:54.278 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.215+0000 7fafc652c700 1 -- 172.21.15.17:0/399508465 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafbc0095f0 con 0x7fafc010c3e0 2023-07-14T12:00:54.278 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.216+0000 7fafc652c700 1 --2- 172.21.15.17:0/399508465 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafc010c3e0 0x7fafc019c7c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fafb8007ee0 tx=0x7fafb800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:54.278 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.216+0000 7fafb6ffd700 1 -- 172.21.15.17:0/399508465 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb800dd20 con 0x7fafc010c3e0 2023-07-14T12:00:54.278 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.216+0000 7fafb6ffd700 1 -- 172.21.15.17:0/399508465 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fafb800f040 con 0x7fafc010c3e0 2023-07-14T12:00:54.278 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.216+0000 7fafc7f8f700 1 -- 172.21.15.17:0/399508465 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafc019d470 con 0x7fafc010c3e0 2023-07-14T12:00:54.278 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.216+0000 7fafb6ffd700 1 -- 172.21.15.17:0/399508465 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb80048a0 con 0x7fafc010c3e0 2023-07-14T12:00:54.278 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.216+0000 7fafc7f8f700 1 -- 172.21.15.17:0/399508465 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafc01a4c90 con 0x7fafc010c3e0 2023-07-14T12:00:54.278 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.217+0000 7fafc7f8f700 1 -- 172.21.15.17:0/399508465 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafc004e210 con 0x7fafc010c3e0 2023-07-14T12:00:54.279 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.219+0000 7fafb6ffd700 1 -- 172.21.15.17:0/399508465 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fafb8003a00 con 0x7fafc010c3e0 2023-07-14T12:00:54.279 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.221+0000 7fafb6ffd700 1 --2- 172.21.15.17:0/399508465 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fafac0a11d0 0x7fafac0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:54.279 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.221+0000 7fafb6ffd700 1 -- 172.21.15.17:0/399508465 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fafb80cc390 con 0x7fafc010c3e0 2023-07-14T12:00:54.279 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.221+0000 7fafc5d2b700 1 --2- 172.21.15.17:0/399508465 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fafac0a11d0 0x7fafac0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:54.279 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.222+0000 7fafc5d2b700 1 --2- 172.21.15.17:0/399508465 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fafac0a11d0 0x7fafac0a3680 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fafbc00b2e0 tx=0x7fafbc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:54.279 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.224+0000 7fafb6ffd700 1 -- 172.21.15.17:0/399508465 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fafb804e790 con 0x7fafc010c3e0 2023-07-14T12:00:54.317 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070502 got 253403070504 for osd.8 2023-07-14T12:00:54.317 DEBUG:teuthology.parallel:result is None 2023-07-14T12:00:54.319 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149756 got 111669149758 for osd.3 2023-07-14T12:00:54.319 DEBUG:teuthology.parallel:result is None 2023-07-14T12:00:54.448 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.468+0000 7fafc7f8f700 1 -- 172.21.15.17:0/399508465 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fafc01a5010 con 0x7fafc010c3e0 2023-07-14T12:00:54.449 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.469+0000 7fafb6ffd700 1 -- 172.21.15.17:0/399508465 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fafb8095770 con 0x7fafc010c3e0 2023-07-14T12:00:54.449 INFO:teuthology.orchestra.run.smithi017.stdout:34359738444 2023-07-14T12:00:54.451 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.471+0000 7fafc7f8f700 1 -- 172.21.15.17:0/399508465 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fafac0a11d0 msgr2=0x7fafac0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:54.451 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.471+0000 7fafc7f8f700 1 --2- 172.21.15.17:0/399508465 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fafac0a11d0 0x7fafac0a3680 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fafbc00b2e0 tx=0x7fafbc005fd0 comp rx=0 tx=0).stop 2023-07-14T12:00:54.452 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.471+0000 7fafc7f8f700 1 -- 172.21.15.17:0/399508465 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafc010c3e0 msgr2=0x7fafc019c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:54.452 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.471+0000 7fafc7f8f700 1 --2- 172.21.15.17:0/399508465 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafc010c3e0 0x7fafc019c7c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fafb8007ee0 tx=0x7fafb800d3b0 comp rx=0 tx=0).stop 2023-07-14T12:00:54.452 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.471+0000 7fafc7f8f700 1 -- 172.21.15.17:0/399508465 shutdown_connections 2023-07-14T12:00:54.452 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.471+0000 7fafc7f8f700 1 --2- 172.21.15.17:0/399508465 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fafac0a11d0 0x7fafac0a3680 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:54.452 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.471+0000 7fafc7f8f700 1 --2- 172.21.15.17:0/399508465 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fafc00728b0 0x7fafc0197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:54.452 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.471+0000 7fafc7f8f700 1 --2- 172.21.15.17:0/399508465 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafc010c3e0 0x7fafc019c7c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:54.452 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.471+0000 7fafc7f8f700 1 --2- 172.21.15.17:0/399508465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafc00731c0 0x7fafc0198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:54.453 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.471+0000 7fafc7f8f700 1 -- 172.21.15.17:0/399508465 >> 172.21.15.17:0/399508465 conn(0x7fafc00fb660 msgr2=0x7fafc010e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:54.453 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.471+0000 7fafc7f8f700 1 -- 172.21.15.17:0/399508465 shutdown_connections 2023-07-14T12:00:54.453 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:54.471+0000 7fafc7f8f700 1 -- 172.21.15.17:0/399508465 wait complete. 2023-07-14T12:00:54.904 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411393 got 77309411395 for osd.2 2023-07-14T12:00:54.905 DEBUG:teuthology.parallel:result is None 2023-07-14T12:00:55.073 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:54 smithi017 ceph-mon[126339]: pgmap v314: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-07-14T12:00:55.074 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:54 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/281593326' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-07-14T12:00:55.074 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:54 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/399508465' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-07-14T12:00:55.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:54 smithi093 ceph-mon[128922]: pgmap v314: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-07-14T12:00:55.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:54 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/281593326' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-07-14T12:00:55.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:54 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/399508465' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-07-14T12:00:55.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:54 smithi097 ceph-mon[129313]: pgmap v314: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-07-14T12:00:55.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:54 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/281593326' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-07-14T12:00:55.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:54 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/399508465' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-07-14T12:00:55.333 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808865 got 287762808866 for osd.9 2023-07-14T12:00:55.334 DEBUG:teuthology.parallel:result is None 2023-07-14T12:00:55.474 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T12:00:55.478 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.497+0000 7f7d46a04700 1 -- 172.21.15.17:0/1408475338 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7d40100df0 msgr2=0x7f7d401011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:55.478 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.497+0000 7f7d46a04700 1 --2- 172.21.15.17:0/1408475338 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7d40100df0 0x7f7d401011c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7d30009990 tx=0x7f7d30009ca0 comp rx=0 tx=0).stop 2023-07-14T12:00:55.478 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.497+0000 7f7d46a04700 1 -- 172.21.15.17:0/1408475338 shutdown_connections 2023-07-14T12:00:55.479 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.497+0000 7f7d46a04700 1 --2- 172.21.15.17:0/1408475338 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7d40100df0 0x7f7d401011c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:55.479 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.497+0000 7f7d46a04700 1 --2- 172.21.15.17:0/1408475338 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d4010a140 0x7f7d4010c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:55.479 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.497+0000 7f7d46a04700 1 --2- 172.21.15.17:0/1408475338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d40101700 0x7f7d40109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:55.479 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.497+0000 7f7d46a04700 1 -- 172.21.15.17:0/1408475338 >> 172.21.15.17:0/1408475338 conn(0x7f7d40075b70 msgr2=0x7f7d40075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:55.480 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.497+0000 7f7d46a04700 1 -- 172.21.15.17:0/1408475338 shutdown_connections 2023-07-14T12:00:55.480 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.498+0000 7f7d46a04700 1 -- 172.21.15.17:0/1408475338 wait complete. 2023-07-14T12:00:55.482 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.499+0000 7f7d46a04700 1 Processor -- start 2023-07-14T12:00:55.482 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.499+0000 7f7d46a04700 1 -- start start 2023-07-14T12:00:55.482 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.500+0000 7f7d46a04700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d40100df0 0x7f7d400fea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:55.483 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.500+0000 7f7d46a04700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7d40101700 0x7f7d400fef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:55.483 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.500+0000 7f7d46a04700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d4010a140 0x7f7d400ff510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:55.483 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.500+0000 7f7d46a04700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d401a4080 con 0x7f7d40100df0 2023-07-14T12:00:55.483 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.500+0000 7f7d46a04700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d401a41f0 con 0x7f7d40101700 2023-07-14T12:00:55.484 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.500+0000 7f7d46a04700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d401a43b0 con 0x7f7d4010a140 2023-07-14T12:00:55.484 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.500+0000 7f7d3f7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7d40101700 0x7f7d400fef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:55.484 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.500+0000 7f7d3ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d40100df0 0x7f7d400fea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:55.485 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.500+0000 7f7d3f7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7d40101700 0x7f7d400fef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.17:42590/0 (socket says 172.21.15.17:42590) 2023-07-14T12:00:55.485 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.500+0000 7f7d3f7fe700 1 -- 172.21.15.17:0/1707725644 learned_addr learned my addr 172.21.15.17:0/1707725644 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:00:55.485 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.500+0000 7f7d44fa1700 1 --2- 172.21.15.17:0/1707725644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d4010a140 0x7f7d400ff510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:55.485 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.501+0000 7f7d3f7fe700 1 -- 172.21.15.17:0/1707725644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d4010a140 msgr2=0x7f7d400ff510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:55.486 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.501+0000 7f7d3f7fe700 1 --2- 172.21.15.17:0/1707725644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d4010a140 0x7f7d400ff510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:55.486 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.501+0000 7f7d3f7fe700 1 -- 172.21.15.17:0/1707725644 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d40100df0 msgr2=0x7f7d400fea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:55.486 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.501+0000 7f7d3f7fe700 1 --2- 172.21.15.17:0/1707725644 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d40100df0 0x7f7d400fea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:55.486 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.501+0000 7f7d3f7fe700 1 -- 172.21.15.17:0/1707725644 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d300095f0 con 0x7f7d40101700 2023-07-14T12:00:55.487 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.501+0000 7f7d3ffff700 1 --2- 172.21.15.17:0/1707725644 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d40100df0 0x7f7d400fea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T12:00:55.487 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.501+0000 7f7d3f7fe700 1 --2- 172.21.15.17:0/1707725644 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7d40101700 0x7f7d400fef90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7d3400e9a0 tx=0x7f7d3400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:55.487 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.501+0000 7f7d3d7fa700 1 -- 172.21.15.17:0/1707725644 <== mon.2 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d34009790 con 0x7f7d40101700 2023-07-14T12:00:55.487 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.501+0000 7f7d3d7fa700 1 -- 172.21.15.17:0/1707725644 <== mon.2 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d34010430 con 0x7f7d40101700 2023-07-14T12:00:55.488 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.501+0000 7f7d3d7fa700 1 -- 172.21.15.17:0/1707725644 <== mon.2 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d34018680 con 0x7f7d40101700 2023-07-14T12:00:55.488 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.502+0000 7f7d46a04700 1 -- 172.21.15.17:0/1707725644 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d401a4690 con 0x7f7d40101700 2023-07-14T12:00:55.488 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.502+0000 7f7d46a04700 1 -- 172.21.15.17:0/1707725644 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d401a4b40 con 0x7f7d40101700 2023-07-14T12:00:55.489 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.503+0000 7f7d46a04700 1 -- 172.21.15.17:0/1707725644 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d4004e210 con 0x7f7d40101700 2023-07-14T12:00:55.490 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.506+0000 7f7d3d7fa700 1 -- 172.21.15.17:0/1707725644 <== mon.2 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7d340105a0 con 0x7f7d40101700 2023-07-14T12:00:55.490 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.507+0000 7f7d3d7fa700 1 --2- 172.21.15.17:0/1707725644 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f7d280a1150 0x7f7d280a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:55.490 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.508+0000 7f7d3d7fa700 1 -- 172.21.15.17:0/1707725644 <== mon.2 v2:172.21.15.93:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7d34014070 con 0x7f7d40101700 2023-07-14T12:00:55.490 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.508+0000 7f7d3ffff700 1 --2- 172.21.15.17:0/1707725644 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f7d280a1150 0x7f7d280a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:55.491 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.509+0000 7f7d3ffff700 1 --2- 172.21.15.17:0/1707725644 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f7d280a1150 0x7f7d280a3600 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7d3000b2e0 tx=0x7f7d30005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:55.491 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.509+0000 7f7d3d7fa700 1 -- 172.21.15.17:0/1707725644 <== mon.2 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7d340961d0 con 0x7f7d40101700 2023-07-14T12:00:55.679 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.571+0000 7f400bbe3700 1 -- 172.21.15.17:0/1376196800 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40040728b0 msgr2=0x7f4004072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:55.680 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.571+0000 7f400bbe3700 1 --2- 172.21.15.17:0/1376196800 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40040728b0 0x7f4004072c80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f40000098f0 tx=0x7f4000009c00 comp rx=0 tx=0).stop 2023-07-14T12:00:55.680 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.572+0000 7f400bbe3700 1 -- 172.21.15.17:0/1376196800 shutdown_connections 2023-07-14T12:00:55.680 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.572+0000 7f400bbe3700 1 --2- 172.21.15.17:0/1376196800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4004110bb0 0x7f4004112f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:55.680 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.572+0000 7f400bbe3700 1 --2- 172.21.15.17:0/1376196800 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40040728b0 0x7f4004072c80 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:55.680 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.572+0000 7f400bbe3700 1 --2- 172.21.15.17:0/1376196800 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f40040731c0 0x7f4004110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:55.680 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.572+0000 7f400bbe3700 1 -- 172.21.15.17:0/1376196800 >> 172.21.15.17:0/1376196800 conn(0x7f40040fb660 msgr2=0x7f40040fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:55.681 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.572+0000 7f400bbe3700 1 -- 172.21.15.17:0/1376196800 shutdown_connections 2023-07-14T12:00:55.681 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.572+0000 7f400bbe3700 1 -- 172.21.15.17:0/1376196800 wait complete. 2023-07-14T12:00:55.681 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.573+0000 7f400bbe3700 1 Processor -- start 2023-07-14T12:00:55.681 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.573+0000 7f400bbe3700 1 -- start start 2023-07-14T12:00:55.681 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.573+0000 7f400bbe3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f40040728b0 0x7f4004071b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:55.681 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.573+0000 7f400bbe3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40040731c0 0x7f4004072040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:55.681 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.573+0000 7f400bbe3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4004110bb0 0x7f400406baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:55.682 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.573+0000 7f400bbe3700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400406c160 con 0x7f40040731c0 2023-07-14T12:00:55.682 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.573+0000 7f400bbe3700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400406c2d0 con 0x7f40040728b0 2023-07-14T12:00:55.682 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.573+0000 7f400bbe3700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400406c490 con 0x7f4004110bb0 2023-07-14T12:00:55.682 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.574+0000 7f400997f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f40040728b0 0x7f4004071b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:55.682 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.574+0000 7f400a180700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4004110bb0 0x7f400406baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:55.682 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.574+0000 7f400a180700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4004110bb0 0x7f400406baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.17:33742/0 (socket says 172.21.15.17:33742) 2023-07-14T12:00:55.682 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.574+0000 7f400a180700 1 -- 172.21.15.17:0/2482738409 learned_addr learned my addr 172.21.15.17:0/2482738409 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:00:55.682 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.574+0000 7f400a180700 1 -- 172.21.15.17:0/2482738409 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f40040728b0 msgr2=0x7f4004071b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:55.683 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.574+0000 7f400a180700 1 --2- 172.21.15.17:0/2482738409 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f40040728b0 0x7f4004071b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:55.683 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.574+0000 7f400917e700 1 --2- 172.21.15.17:0/2482738409 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40040731c0 0x7f4004072040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:55.683 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.574+0000 7f400a180700 1 -- 172.21.15.17:0/2482738409 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40040731c0 msgr2=0x7f4004072040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:55.683 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.574+0000 7f400a180700 1 --2- 172.21.15.17:0/2482738409 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40040731c0 0x7f4004072040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:55.683 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.574+0000 7f400a180700 1 -- 172.21.15.17:0/2482738409 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4000009520 con 0x7f4004110bb0 2023-07-14T12:00:55.683 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.574+0000 7f400917e700 1 --2- 172.21.15.17:0/2482738409 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40040731c0 0x7f4004072040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T12:00:55.683 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.575+0000 7f400a180700 1 --2- 172.21.15.17:0/2482738409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4004110bb0 0x7f400406baf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc00ef90 tx=0x7f3ffc00c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:55.684 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.575+0000 7f3ffaffd700 1 -- 172.21.15.17:0/2482738409 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc018070 con 0x7f4004110bb0 2023-07-14T12:00:55.684 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.575+0000 7f400bbe3700 1 -- 172.21.15.17:0/2482738409 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f400406c710 con 0x7f4004110bb0 2023-07-14T12:00:55.684 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.575+0000 7f400bbe3700 1 -- 172.21.15.17:0/2482738409 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40041b18a0 con 0x7f4004110bb0 2023-07-14T12:00:55.684 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.575+0000 7f3ffaffd700 1 -- 172.21.15.17:0/2482738409 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ffc00f040 con 0x7f4004110bb0 2023-07-14T12:00:55.684 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.575+0000 7f3ffaffd700 1 -- 172.21.15.17:0/2482738409 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc003ea0 con 0x7f4004110bb0 2023-07-14T12:00:55.684 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.578+0000 7f400bbe3700 1 -- 172.21.15.17:0/2482738409 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fe8005320 con 0x7f4004110bb0 2023-07-14T12:00:55.684 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.579+0000 7f3ffaffd700 1 -- 172.21.15.17:0/2482738409 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3ffc00cb00 con 0x7f4004110bb0 2023-07-14T12:00:55.685 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.580+0000 7f3ffaffd700 1 --2- 172.21.15.17:0/2482738409 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3ff00a1220 0x7f3ff00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:55.685 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.580+0000 7f3ffaffd700 1 -- 172.21.15.17:0/2482738409 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3ffc0cc760 con 0x7f4004110bb0 2023-07-14T12:00:55.685 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.581+0000 7f400997f700 1 --2- 172.21.15.17:0/2482738409 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3ff00a1220 0x7f3ff00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:55.685 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.582+0000 7f400997f700 1 --2- 172.21.15.17:0/2482738409 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3ff00a1220 0x7f3ff00a36d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4000000c00 tx=0x7f40000058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:55.685 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.584+0000 7f3ffaffd700 1 -- 172.21.15.17:0/2482738409 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3ffc095bd0 con 0x7f4004110bb0 2023-07-14T12:00:55.719 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.739+0000 7f7d46a04700 1 -- 172.21.15.17:0/1707725644 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f7d400fff80 con 0x7f7d40101700 2023-07-14T12:00:55.721 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.741+0000 7f7d3d7fa700 1 -- 172.21.15.17:0/1707725644 <== mon.2 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7d340997f0 con 0x7f7d40101700 2023-07-14T12:00:55.722 INFO:teuthology.orchestra.run.smithi017.stdout:223338299437 2023-07-14T12:00:55.725 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.744+0000 7f7d46a04700 1 -- 172.21.15.17:0/1707725644 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f7d280a1150 msgr2=0x7f7d280a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:55.725 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.744+0000 7f7d46a04700 1 --2- 172.21.15.17:0/1707725644 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f7d280a1150 0x7f7d280a3600 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7d3000b2e0 tx=0x7f7d30005960 comp rx=0 tx=0).stop 2023-07-14T12:00:55.725 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.744+0000 7f7d46a04700 1 -- 172.21.15.17:0/1707725644 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7d40101700 msgr2=0x7f7d400fef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:55.725 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.744+0000 7f7d46a04700 1 --2- 172.21.15.17:0/1707725644 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7d40101700 0x7f7d400fef90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7d3400e9a0 tx=0x7f7d3400ecb0 comp rx=0 tx=0).stop 2023-07-14T12:00:55.725 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.744+0000 7f7d46a04700 1 -- 172.21.15.17:0/1707725644 shutdown_connections 2023-07-14T12:00:55.725 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.744+0000 7f7d46a04700 1 --2- 172.21.15.17:0/1707725644 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f7d280a1150 0x7f7d280a3600 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:55.726 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.744+0000 7f7d46a04700 1 --2- 172.21.15.17:0/1707725644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d4010a140 0x7f7d400ff510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:55.726 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.744+0000 7f7d46a04700 1 --2- 172.21.15.17:0/1707725644 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d40100df0 0x7f7d400fea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:55.726 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.744+0000 7f7d46a04700 1 --2- 172.21.15.17:0/1707725644 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7d40101700 0x7f7d400fef90 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:55.726 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.744+0000 7f7d46a04700 1 -- 172.21.15.17:0/1707725644 >> 172.21.15.17:0/1707725644 conn(0x7f7d40075b70 msgr2=0x7f7d400fd300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:55.726 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.745+0000 7f7d46a04700 1 -- 172.21.15.17:0/1707725644 shutdown_connections 2023-07-14T12:00:55.726 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.745+0000 7f7d46a04700 1 -- 172.21.15.17:0/1707725644 wait complete. 2023-07-14T12:00:55.793 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.813+0000 7f400bbe3700 1 -- 172.21.15.17:0/2482738409 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f3fe8005190 con 0x7f4004110bb0 2023-07-14T12:00:55.793 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.813+0000 7f3ffaffd700 1 -- 172.21.15.17:0/2482738409 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3ffc00cdb0 con 0x7f4004110bb0 2023-07-14T12:00:55.794 INFO:teuthology.orchestra.run.smithi017.stdout:167503724599 2023-07-14T12:00:55.796 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.816+0000 7f400bbe3700 1 -- 172.21.15.17:0/2482738409 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3ff00a1220 msgr2=0x7f3ff00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:55.796 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.816+0000 7f400bbe3700 1 --2- 172.21.15.17:0/2482738409 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3ff00a1220 0x7f3ff00a36d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4000000c00 tx=0x7f40000058e0 comp rx=0 tx=0).stop 2023-07-14T12:00:55.797 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.816+0000 7f400bbe3700 1 -- 172.21.15.17:0/2482738409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4004110bb0 msgr2=0x7f400406baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:55.797 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.816+0000 7f400bbe3700 1 --2- 172.21.15.17:0/2482738409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4004110bb0 0x7f400406baf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc00ef90 tx=0x7f3ffc00c450 comp rx=0 tx=0).stop 2023-07-14T12:00:55.797 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.816+0000 7f400bbe3700 1 -- 172.21.15.17:0/2482738409 shutdown_connections 2023-07-14T12:00:55.797 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.816+0000 7f400bbe3700 1 --2- 172.21.15.17:0/2482738409 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3ff00a1220 0x7f3ff00a36d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:55.797 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.816+0000 7f400bbe3700 1 --2- 172.21.15.17:0/2482738409 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f40040728b0 0x7f4004071b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:55.797 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.816+0000 7f400bbe3700 1 --2- 172.21.15.17:0/2482738409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4004110bb0 0x7f400406baf0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:55.797 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.816+0000 7f400bbe3700 1 --2- 172.21.15.17:0/2482738409 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40040731c0 0x7f4004072040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:55.798 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.816+0000 7f400bbe3700 1 -- 172.21.15.17:0/2482738409 >> 172.21.15.17:0/2482738409 conn(0x7f40040fb660 msgr2=0x7f40040fc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:55.798 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.816+0000 7f400bbe3700 1 -- 172.21.15.17:0/2482738409 shutdown_connections 2023-07-14T12:00:55.798 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:55.816+0000 7f400bbe3700 1 -- 172.21.15.17:0/2482738409 wait complete. 2023-07-14T12:00:56.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:55 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/1707725644' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-07-14T12:00:56.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:55 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/1707725644' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-07-14T12:00:56.143 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:55 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/1707725644' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-07-14T12:00:56.189 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T12:00:56.894 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:56 smithi017 ceph-mon[126339]: pgmap v315: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-07-14T12:00:56.894 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:56 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/2482738409' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-07-14T12:00:57.033 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724596 got 167503724599 for osd.5 2023-07-14T12:00:57.033 DEBUG:teuthology.parallel:result is None 2023-07-14T12:00:57.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:56 smithi093 ceph-mon[128922]: pgmap v315: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-07-14T12:00:57.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:56 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/2482738409' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-07-14T12:00:57.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:56 smithi097 ceph-mon[129313]: pgmap v315: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-07-14T12:00:57.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:56 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/2482738409' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-07-14T12:00:57.415 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299435 got 223338299437 for osd.7 2023-07-14T12:00:57.415 DEBUG:teuthology.parallel:result is None 2023-07-14T12:00:57.456 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738440 got 34359738444 for osd.0 2023-07-14T12:00:57.456 DEBUG:teuthology.parallel:result is None 2023-07-14T12:00:58.486 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.505+0000 7fed0f8f3700 1 -- 172.21.15.17:0/3291149702 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed080728b0 msgr2=0x7fed08072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:58.486 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.505+0000 7fed0f8f3700 1 --2- 172.21.15.17:0/3291149702 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed080728b0 0x7fed08072c80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fed04009990 tx=0x7fed04009ca0 comp rx=0 tx=0).stop 2023-07-14T12:00:58.487 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.505+0000 7fed0f8f3700 1 -- 172.21.15.17:0/3291149702 shutdown_connections 2023-07-14T12:00:58.487 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.505+0000 7fed0f8f3700 1 --2- 172.21.15.17:0/3291149702 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed08110bb0 0x7fed08112f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:58.487 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.505+0000 7fed0f8f3700 1 --2- 172.21.15.17:0/3291149702 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed080728b0 0x7fed08072c80 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:58.487 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.506+0000 7fed0f8f3700 1 --2- 172.21.15.17:0/3291149702 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fed080731c0 0x7fed08110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:58.487 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.506+0000 7fed0f8f3700 1 -- 172.21.15.17:0/3291149702 >> 172.21.15.17:0/3291149702 conn(0x7fed080fb660 msgr2=0x7fed080fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:58.487 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.506+0000 7fed0f8f3700 1 -- 172.21.15.17:0/3291149702 shutdown_connections 2023-07-14T12:00:58.487 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.506+0000 7fed0f8f3700 1 -- 172.21.15.17:0/3291149702 wait complete. 2023-07-14T12:00:58.488 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.507+0000 7fed0f8f3700 1 Processor -- start 2023-07-14T12:00:58.488 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.508+0000 7fed0f8f3700 1 -- start start 2023-07-14T12:00:58.489 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.508+0000 7fed0f8f3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed080728b0 0x7fed0819c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:58.489 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.508+0000 7fed0f8f3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fed080731c0 0x7fed0819c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:58.489 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.508+0000 7fed0f8f3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed08110bb0 0x7fed081a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:58.489 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.508+0000 7fed0f8f3700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed081a1280 con 0x7fed08110bb0 2023-07-14T12:00:58.489 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.508+0000 7fed0f8f3700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed081a13f0 con 0x7fed080731c0 2023-07-14T12:00:58.489 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.508+0000 7fed0f8f3700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed081a15b0 con 0x7fed080728b0 2023-07-14T12:00:58.490 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.509+0000 7fed0d68f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed080728b0 0x7fed0819c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:58.490 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.509+0000 7fed0d68f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed080728b0 0x7fed0819c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.17:33756/0 (socket says 172.21.15.17:33756) 2023-07-14T12:00:58.490 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.509+0000 7fed0d68f700 1 -- 172.21.15.17:0/2686968334 learned_addr learned my addr 172.21.15.17:0/2686968334 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:00:58.491 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.509+0000 7fed0ce8e700 1 --2- 172.21.15.17:0/2686968334 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fed080731c0 0x7fed0819c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:58.491 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.509+0000 7fed0de90700 1 --2- 172.21.15.17:0/2686968334 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed08110bb0 0x7fed081a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:58.491 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.509+0000 7fed0d68f700 1 -- 172.21.15.17:0/2686968334 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fed080731c0 msgr2=0x7fed0819c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:58.491 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.509+0000 7fed0d68f700 1 --2- 172.21.15.17:0/2686968334 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fed080731c0 0x7fed0819c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:58.491 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.509+0000 7fed0d68f700 1 -- 172.21.15.17:0/2686968334 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed08110bb0 msgr2=0x7fed081a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:58.491 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.509+0000 7fed0d68f700 1 --2- 172.21.15.17:0/2686968334 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed08110bb0 0x7fed081a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:58.491 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.509+0000 7fed0d68f700 1 -- 172.21.15.17:0/2686968334 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed040095f0 con 0x7fed080728b0 2023-07-14T12:00:58.491 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.509+0000 7fed0d68f700 1 --2- 172.21.15.17:0/2686968334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed080728b0 0x7fed0819c2c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fed04005910 tx=0x7fed04005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:58.492 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.509+0000 7fecfe7fc700 1 -- 172.21.15.17:0/2686968334 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0401d070 con 0x7fed080728b0 2023-07-14T12:00:58.492 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.510+0000 7fecfe7fc700 1 -- 172.21.15.17:0/2686968334 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed04022470 con 0x7fed080728b0 2023-07-14T12:00:58.492 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.510+0000 7fecfe7fc700 1 -- 172.21.15.17:0/2686968334 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed04017650 con 0x7fed080728b0 2023-07-14T12:00:58.492 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.510+0000 7fed0f8f3700 1 -- 172.21.15.17:0/2686968334 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed081a1830 con 0x7fed080728b0 2023-07-14T12:00:58.492 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.510+0000 7fed0f8f3700 1 -- 172.21.15.17:0/2686968334 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed081a8f20 con 0x7fed080728b0 2023-07-14T12:00:58.493 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.511+0000 7fed0f8f3700 1 -- 172.21.15.17:0/2686968334 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed08103090 con 0x7fed080728b0 2023-07-14T12:00:58.493 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.513+0000 7fecfe7fc700 1 -- 172.21.15.17:0/2686968334 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fed040225e0 con 0x7fed080728b0 2023-07-14T12:00:58.495 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.515+0000 7fecfe7fc700 1 --2- 172.21.15.17:0/2686968334 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fecf40a11d0 0x7fecf40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:58.495 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.515+0000 7fecfe7fc700 1 -- 172.21.15.17:0/2686968334 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fed040cdeb0 con 0x7fed080728b0 2023-07-14T12:00:58.496 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.515+0000 7fed0ce8e700 1 --2- 172.21.15.17:0/2686968334 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fecf40a11d0 0x7fecf40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:58.496 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.516+0000 7fed0ce8e700 1 --2- 172.21.15.17:0/2686968334 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fecf40a11d0 0x7fecf40a3680 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fed0819d8e0 tx=0x7fecf8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:58.497 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.517+0000 7fecfe7fc700 1 -- 172.21.15.17:0/2686968334 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fed04097320 con 0x7fed080728b0 2023-07-14T12:00:58.567 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T12:00:58.762 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.782+0000 7fed0f8f3700 1 -- 172.21.15.17:0/2686968334 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fed0804e210 con 0x7fed080728b0 2023-07-14T12:00:58.800 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.783+0000 7fecfe7fc700 1 -- 172.21.15.17:0/2686968334 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fed0409a940 con 0x7fed080728b0 2023-07-14T12:00:58.801 INFO:teuthology.orchestra.run.smithi017.stdout:55834574922 2023-07-14T12:00:58.801 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.785+0000 7fed0f8f3700 1 -- 172.21.15.17:0/2686968334 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fecf40a11d0 msgr2=0x7fecf40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:58.801 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.785+0000 7fed0f8f3700 1 --2- 172.21.15.17:0/2686968334 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fecf40a11d0 0x7fecf40a3680 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fed0819d8e0 tx=0x7fecf8009260 comp rx=0 tx=0).stop 2023-07-14T12:00:58.801 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.785+0000 7fed0f8f3700 1 -- 172.21.15.17:0/2686968334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed080728b0 msgr2=0x7fed0819c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:58.802 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.785+0000 7fed0f8f3700 1 --2- 172.21.15.17:0/2686968334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed080728b0 0x7fed0819c2c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fed04005910 tx=0x7fed04005720 comp rx=0 tx=0).stop 2023-07-14T12:00:58.802 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.785+0000 7fed0f8f3700 1 -- 172.21.15.17:0/2686968334 shutdown_connections 2023-07-14T12:00:58.802 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.785+0000 7fed0f8f3700 1 --2- 172.21.15.17:0/2686968334 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fecf40a11d0 0x7fecf40a3680 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:58.802 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.785+0000 7fed0f8f3700 1 --2- 172.21.15.17:0/2686968334 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed08110bb0 0x7fed081a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:58.802 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.785+0000 7fed0f8f3700 1 --2- 172.21.15.17:0/2686968334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed080728b0 0x7fed0819c2c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:58.802 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.785+0000 7fed0f8f3700 1 --2- 172.21.15.17:0/2686968334 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fed080731c0 0x7fed0819c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:58.802 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.785+0000 7fed0f8f3700 1 -- 172.21.15.17:0/2686968334 >> 172.21.15.17:0/2686968334 conn(0x7fed080fb660 msgr2=0x7fed080fc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:58.803 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.786+0000 7fed0f8f3700 1 -- 172.21.15.17:0/2686968334 shutdown_connections 2023-07-14T12:00:58.803 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:58.786+0000 7fed0f8f3700 1 -- 172.21.15.17:0/2686968334 wait complete. 2023-07-14T12:00:58.918 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:58 smithi017 ceph-mon[126339]: pgmap v316: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2023-07-14T12:00:58.918 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:58 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/2686968334' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-07-14T12:00:59.026 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.045+0000 7fb0144a2700 1 -- 172.21.15.17:0/1401493179 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb00c0728b0 msgr2=0x7fb00c072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:59.026 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.045+0000 7fb0144a2700 1 --2- 172.21.15.17:0/1401493179 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb00c0728b0 0x7fb00c072c80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb008009990 tx=0x7fb008009ca0 comp rx=0 tx=0).stop 2023-07-14T12:00:59.026 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.046+0000 7fb0144a2700 1 -- 172.21.15.17:0/1401493179 shutdown_connections 2023-07-14T12:00:59.026 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.046+0000 7fb0144a2700 1 --2- 172.21.15.17:0/1401493179 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb00c10c3e0 0x7fb00c10e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:59.027 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.046+0000 7fb0144a2700 1 --2- 172.21.15.17:0/1401493179 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb00c0728b0 0x7fb00c072c80 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:59.027 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.046+0000 7fb0144a2700 1 --2- 172.21.15.17:0/1401493179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb00c0731c0 0x7fb00c10bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:59.027 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.046+0000 7fb0144a2700 1 -- 172.21.15.17:0/1401493179 >> 172.21.15.17:0/1401493179 conn(0x7fb00c0fb660 msgr2=0x7fb00c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:59.027 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.046+0000 7fb0144a2700 1 -- 172.21.15.17:0/1401493179 shutdown_connections 2023-07-14T12:00:59.027 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.046+0000 7fb0144a2700 1 -- 172.21.15.17:0/1401493179 wait complete. 2023-07-14T12:00:59.028 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.047+0000 7fb0144a2700 1 Processor -- start 2023-07-14T12:00:59.029 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.047+0000 7fb0144a2700 1 -- start start 2023-07-14T12:00:59.029 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.047+0000 7fb0144a2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb00c0728b0 0x7fb00c197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:59.029 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.047+0000 7fb0144a2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb00c0731c0 0x7fb00c198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:59.029 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.047+0000 7fb0144a2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb00c10c3e0 0x7fb00c19c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:59.029 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.047+0000 7fb0144a2700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00c19ce60 con 0x7fb00c10c3e0 2023-07-14T12:00:59.029 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.047+0000 7fb0144a2700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00c19cfd0 con 0x7fb00c0728b0 2023-07-14T12:00:59.029 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.047+0000 7fb0144a2700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00c19d190 con 0x7fb00c0731c0 2023-07-14T12:00:59.030 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.048+0000 7fb01223e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb00c0728b0 0x7fb00c197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:59.030 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.048+0000 7fb01223e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb00c0728b0 0x7fb00c197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.17:42630/0 (socket says 172.21.15.17:42630) 2023-07-14T12:00:59.030 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.048+0000 7fb01223e700 1 -- 172.21.15.17:0/1157073740 learned_addr learned my addr 172.21.15.17:0/1157073740 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:00:59.030 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.048+0000 7fb011a3d700 1 --2- 172.21.15.17:0/1157073740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb00c0731c0 0x7fb00c198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:59.030 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.048+0000 7fb012a3f700 1 --2- 172.21.15.17:0/1157073740 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb00c10c3e0 0x7fb00c19c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:59.030 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.048+0000 7fb01223e700 1 -- 172.21.15.17:0/1157073740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb00c0731c0 msgr2=0x7fb00c198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:59.030 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.048+0000 7fb01223e700 1 --2- 172.21.15.17:0/1157073740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb00c0731c0 0x7fb00c198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:59.030 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.048+0000 7fb01223e700 1 -- 172.21.15.17:0/1157073740 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb00c10c3e0 msgr2=0x7fb00c19c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:59.031 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.048+0000 7fb01223e700 1 --2- 172.21.15.17:0/1157073740 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb00c10c3e0 0x7fb00c19c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:59.031 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.048+0000 7fb01223e700 1 -- 172.21.15.17:0/1157073740 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faffc009520 con 0x7fb00c0728b0 2023-07-14T12:00:59.031 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.048+0000 7fb012a3f700 1 --2- 172.21.15.17:0/1157073740 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb00c10c3e0 0x7fb00c19c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T12:00:59.031 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.048+0000 7fb01223e700 1 --2- 172.21.15.17:0/1157073740 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb00c0728b0 0x7fb00c197e10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb008005910 tx=0x7fb008005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:59.031 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.049+0000 7fb0037fe700 1 -- 172.21.15.17:0/1157073740 <== mon.2 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00801d070 con 0x7fb00c0728b0 2023-07-14T12:00:59.032 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.049+0000 7fb0037fe700 1 -- 172.21.15.17:0/1157073740 <== mon.2 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb008022470 con 0x7fb00c0728b0 2023-07-14T12:00:59.032 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.049+0000 7fb0144a2700 1 -- 172.21.15.17:0/1157073740 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0080095f0 con 0x7fb00c0728b0 2023-07-14T12:00:59.032 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.049+0000 7fb0037fe700 1 -- 172.21.15.17:0/1157073740 <== mon.2 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb008017650 con 0x7fb00c0728b0 2023-07-14T12:00:59.032 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.049+0000 7fb0144a2700 1 -- 172.21.15.17:0/1157073740 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb00c1a4a70 con 0x7fb00c0728b0 2023-07-14T12:00:59.032 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.050+0000 7fb0144a2700 1 -- 172.21.15.17:0/1157073740 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb00c04e210 con 0x7fb00c0728b0 2023-07-14T12:00:59.033 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.052+0000 7fb0037fe700 1 -- 172.21.15.17:0/1157073740 <== mon.2 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb0080225e0 con 0x7fb00c0728b0 2023-07-14T12:00:59.034 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.054+0000 7fb0037fe700 1 --2- 172.21.15.17:0/1157073740 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7faff80a1150 0x7faff80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:00:59.034 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.054+0000 7fb0037fe700 1 -- 172.21.15.17:0/1157073740 <== mon.2 v2:172.21.15.93:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb0080ce1c0 con 0x7fb00c0728b0 2023-07-14T12:00:59.034 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.054+0000 7fb011a3d700 1 --2- 172.21.15.17:0/1157073740 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7faff80a1150 0x7faff80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:00:59.035 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.055+0000 7fb011a3d700 1 --2- 172.21.15.17:0/1157073740 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7faff80a1150 0x7faff80a3600 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb00c199430 tx=0x7faffc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:00:59.035 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.055+0000 7fb0037fe700 1 -- 172.21.15.17:0/1157073740 <== mon.2 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb008097630 con 0x7fb00c0728b0 2023-07-14T12:00:59.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:58 smithi093 ceph-mon[128922]: pgmap v316: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2023-07-14T12:00:59.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:58 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/2686968334' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-07-14T12:00:59.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:58 smithi097 ceph-mon[129313]: pgmap v316: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2023-07-14T12:00:59.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:58 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/2686968334' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-07-14T12:00:59.254 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.273+0000 7fb0144a2700 1 -- 172.21.15.17:0/1157073740 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fb00c1a4cd0 con 0x7fb00c0728b0 2023-07-14T12:00:59.255 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.274+0000 7fb0037fe700 1 -- 172.21.15.17:0/1157073740 <== mon.2 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb00809ac50 con 0x7fb00c0728b0 2023-07-14T12:00:59.255 INFO:teuthology.orchestra.run.smithi017.stdout:146028888123 2023-07-14T12:00:59.257 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.277+0000 7fb0144a2700 1 -- 172.21.15.17:0/1157073740 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7faff80a1150 msgr2=0x7faff80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:59.258 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.277+0000 7fb0144a2700 1 --2- 172.21.15.17:0/1157073740 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7faff80a1150 0x7faff80a3600 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb00c199430 tx=0x7faffc009260 comp rx=0 tx=0).stop 2023-07-14T12:00:59.258 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.277+0000 7fb0144a2700 1 -- 172.21.15.17:0/1157073740 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb00c0728b0 msgr2=0x7fb00c197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:00:59.258 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.277+0000 7fb0144a2700 1 --2- 172.21.15.17:0/1157073740 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb00c0728b0 0x7fb00c197e10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb008005910 tx=0x7fb008005720 comp rx=0 tx=0).stop 2023-07-14T12:00:59.258 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.277+0000 7fb0144a2700 1 -- 172.21.15.17:0/1157073740 shutdown_connections 2023-07-14T12:00:59.258 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.277+0000 7fb0144a2700 1 --2- 172.21.15.17:0/1157073740 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7faff80a1150 0x7faff80a3600 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:59.258 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.277+0000 7fb0144a2700 1 --2- 172.21.15.17:0/1157073740 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb00c0728b0 0x7fb00c197e10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:59.259 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.277+0000 7fb0144a2700 1 --2- 172.21.15.17:0/1157073740 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb00c10c3e0 0x7fb00c19c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:59.259 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.277+0000 7fb0144a2700 1 --2- 172.21.15.17:0/1157073740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb00c0731c0 0x7fb00c198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:00:59.259 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.277+0000 7fb0144a2700 1 -- 172.21.15.17:0/1157073740 >> 172.21.15.17:0/1157073740 conn(0x7fb00c0fb660 msgr2=0x7fb00c10e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:00:59.259 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.277+0000 7fb0144a2700 1 -- 172.21.15.17:0/1157073740 shutdown_connections 2023-07-14T12:00:59.259 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:00:59.277+0000 7fb0144a2700 1 -- 172.21.15.17:0/1157073740 wait complete. 2023-07-14T12:00:59.967 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:59 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/1157073740' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-07-14T12:00:59.967 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:59 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T12:00:59.967 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:00:59 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T12:01:00.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:59 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/1157073740' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-07-14T12:01:00.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:59 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T12:01:00.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:00:59 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T12:01:00.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:59 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/1157073740' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-07-14T12:01:00.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:59 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T12:01:00.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:00:59 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T12:01:00.167 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888120 got 146028888123 for osd.4 2023-07-14T12:01:00.167 DEBUG:teuthology.parallel:result is None 2023-07-14T12:01:00.862 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574919 got 55834574922 for osd.1 2023-07-14T12:01:00.862 DEBUG:teuthology.parallel:result is None 2023-07-14T12:01:01.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:00 smithi093 ceph-mon[128922]: pgmap v317: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-07-14T12:01:01.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:00 smithi097 ceph-mon[129313]: pgmap v317: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-07-14T12:01:01.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:00 smithi017 ceph-mon[126339]: pgmap v317: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-07-14T12:01:01.224 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.243+0000 7f8ee2469700 1 -- 172.21.15.17:0/942294616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8edc1007a0 msgr2=0x7f8edc104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:01.224 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.243+0000 7f8ee2469700 1 --2- 172.21.15.17:0/942294616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8edc1007a0 0x7f8edc104770 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8ed00099c0 tx=0x7f8ed0009cd0 comp rx=0 tx=0).stop 2023-07-14T12:01:01.224 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.243+0000 7f8ee2469700 1 -- 172.21.15.17:0/942294616 shutdown_connections 2023-07-14T12:01:01.225 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.243+0000 7f8ee2469700 1 --2- 172.21.15.17:0/942294616 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8edc0ffe00 0x7f8edc1001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:01.225 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.243+0000 7f8ee2469700 1 --2- 172.21.15.17:0/942294616 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8edc104ea0 0x7f8edc10ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:01.225 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.243+0000 7f8ee2469700 1 --2- 172.21.15.17:0/942294616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8edc1007a0 0x7f8edc104770 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:01.225 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.243+0000 7f8ee2469700 1 -- 172.21.15.17:0/942294616 >> 172.21.15.17:0/942294616 conn(0x7f8edc0fb660 msgr2=0x7f8edc0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:01.225 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.243+0000 7f8ee2469700 1 -- 172.21.15.17:0/942294616 shutdown_connections 2023-07-14T12:01:01.225 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.243+0000 7f8ee2469700 1 -- 172.21.15.17:0/942294616 wait complete. 2023-07-14T12:01:01.226 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.245+0000 7f8ee2469700 1 Processor -- start 2023-07-14T12:01:01.226 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.245+0000 7f8ee2469700 1 -- start start 2023-07-14T12:01:01.226 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.246+0000 7f8ee2469700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8edc0ffe00 0x7f8edc197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:01.226 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.246+0000 7f8ee2469700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8edc1007a0 0x7f8edc198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:01.227 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.246+0000 7f8ee2469700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8edc104ea0 0x7f8edc19c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:01.227 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.246+0000 7f8ee2469700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc19cef0 con 0x7f8edc1007a0 2023-07-14T12:01:01.227 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.246+0000 7f8ee2469700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc19d060 con 0x7f8edc104ea0 2023-07-14T12:01:01.227 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.246+0000 7f8ee2469700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc19d220 con 0x7f8edc0ffe00 2023-07-14T12:01:01.228 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.246+0000 7f8edbfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8edc0ffe00 0x7f8edc197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:01.228 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.246+0000 7f8edbfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8edc0ffe00 0x7f8edc197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.17:33816/0 (socket says 172.21.15.17:33816) 2023-07-14T12:01:01.228 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.246+0000 7f8edbfff700 1 -- 172.21.15.17:0/2463804784 learned_addr learned my addr 172.21.15.17:0/2463804784 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:01.228 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.246+0000 7f8edb7fe700 1 --2- 172.21.15.17:0/2463804784 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8edc1007a0 0x7f8edc198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:01.229 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.246+0000 7f8ee0a06700 1 --2- 172.21.15.17:0/2463804784 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8edc104ea0 0x7f8edc19c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:01.229 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.247+0000 7f8edbfff700 1 -- 172.21.15.17:0/2463804784 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8edc104ea0 msgr2=0x7f8edc19c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:01.229 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.247+0000 7f8edbfff700 1 --2- 172.21.15.17:0/2463804784 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8edc104ea0 0x7f8edc19c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:01.229 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.247+0000 7f8edbfff700 1 -- 172.21.15.17:0/2463804784 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8edc1007a0 msgr2=0x7f8edc198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:01.229 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.247+0000 7f8edbfff700 1 --2- 172.21.15.17:0/2463804784 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8edc1007a0 0x7f8edc198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:01.229 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.247+0000 7f8edbfff700 1 -- 172.21.15.17:0/2463804784 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed00095f0 con 0x7f8edc0ffe00 2023-07-14T12:01:01.229 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.247+0000 7f8edb7fe700 1 --2- 172.21.15.17:0/2463804784 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8edc1007a0 0x7f8edc198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T12:01:01.229 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.247+0000 7f8edbfff700 1 --2- 172.21.15.17:0/2463804784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8edc0ffe00 0x7f8edc197f30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc00e9a0 tx=0x7f8ecc00ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:01.230 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.247+0000 7f8ed97fa700 1 -- 172.21.15.17:0/2463804784 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc009790 con 0x7f8edc0ffe00 2023-07-14T12:01:01.230 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.247+0000 7f8ed97fa700 1 -- 172.21.15.17:0/2463804784 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ecc00cbd0 con 0x7f8edc0ffe00 2023-07-14T12:01:01.230 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.247+0000 7f8ee2469700 1 -- 172.21.15.17:0/2463804784 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8edc19d500 con 0x7f8edc0ffe00 2023-07-14T12:01:01.230 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.247+0000 7f8ed97fa700 1 -- 172.21.15.17:0/2463804784 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc018680 con 0x7f8edc0ffe00 2023-07-14T12:01:01.230 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.248+0000 7f8ee2469700 1 -- 172.21.15.17:0/2463804784 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8edc1a4d60 con 0x7f8edc0ffe00 2023-07-14T12:01:01.231 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.249+0000 7f8ee2469700 1 -- 172.21.15.17:0/2463804784 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8edc04e210 con 0x7f8edc0ffe00 2023-07-14T12:01:01.233 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.253+0000 7f8ed97fa700 1 -- 172.21.15.17:0/2463804784 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8ecc00cd40 con 0x7f8edc0ffe00 2023-07-14T12:01:01.235 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.254+0000 7f8ed97fa700 1 --2- 172.21.15.17:0/2463804784 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f8ec40a1150 0x7f8ec40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:01.235 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.255+0000 7f8ed97fa700 1 -- 172.21.15.17:0/2463804784 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8ecc014070 con 0x7f8edc0ffe00 2023-07-14T12:01:01.235 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.255+0000 7f8edb7fe700 1 --2- 172.21.15.17:0/2463804784 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f8ec40a1150 0x7f8ec40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:01.236 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.256+0000 7f8ed97fa700 1 -- 172.21.15.17:0/2463804784 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8ecc09a600 con 0x7f8edc0ffe00 2023-07-14T12:01:01.236 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.256+0000 7f8edb7fe700 1 --2- 172.21.15.17:0/2463804784 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f8ec40a1150 0x7f8ec40a3600 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0009990 tx=0x7f8ed001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:01.301 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.319+0000 7feb8ede8700 1 -- 172.21.15.17:0/58319011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb881007a0 msgr2=0x7feb88104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:01.302 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.319+0000 7feb8ede8700 1 --2- 172.21.15.17:0/58319011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb881007a0 0x7feb88104770 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7feb740099c0 tx=0x7feb74009cd0 comp rx=0 tx=0).stop 2023-07-14T12:01:01.302 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.321+0000 7feb8ede8700 1 -- 172.21.15.17:0/58319011 shutdown_connections 2023-07-14T12:01:01.302 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.321+0000 7feb8ede8700 1 --2- 172.21.15.17:0/58319011 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb880ffe00 0x7feb881001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:01.303 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.321+0000 7feb8ede8700 1 --2- 172.21.15.17:0/58319011 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb88104ea0 0x7feb8810ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:01.303 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.321+0000 7feb8ede8700 1 --2- 172.21.15.17:0/58319011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb881007a0 0x7feb88104770 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:01.303 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.321+0000 7feb8ede8700 1 -- 172.21.15.17:0/58319011 >> 172.21.15.17:0/58319011 conn(0x7feb880fb660 msgr2=0x7feb880fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:01.303 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.321+0000 7feb8ede8700 1 -- 172.21.15.17:0/58319011 shutdown_connections 2023-07-14T12:01:01.303 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.321+0000 7feb8ede8700 1 -- 172.21.15.17:0/58319011 wait complete. 2023-07-14T12:01:01.306 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.323+0000 7feb8ede8700 1 Processor -- start 2023-07-14T12:01:01.306 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.323+0000 7feb8ede8700 1 -- start start 2023-07-14T12:01:01.307 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.323+0000 7feb8ede8700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb880ffe00 0x7feb8806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:01.307 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.323+0000 7feb8ede8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb881007a0 0x7feb8806ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:01.307 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.323+0000 7feb8ede8700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb88104ea0 0x7feb8806b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:01.307 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.323+0000 7feb8ede8700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb88072ca0 con 0x7feb881007a0 2023-07-14T12:01:01.307 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.323+0000 7feb8ede8700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb88072e10 con 0x7feb880ffe00 2023-07-14T12:01:01.307 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.323+0000 7feb8ede8700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb88072fd0 con 0x7feb88104ea0 2023-07-14T12:01:01.307 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.324+0000 7feb8cb84700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb880ffe00 0x7feb8806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:01.307 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.324+0000 7feb8d385700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb88104ea0 0x7feb8806b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:01.308 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.324+0000 7feb8d385700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb88104ea0 0x7feb8806b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.17:33826/0 (socket says 172.21.15.17:33826) 2023-07-14T12:01:01.308 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.324+0000 7feb8d385700 1 -- 172.21.15.17:0/3404921324 learned_addr learned my addr 172.21.15.17:0/3404921324 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:01.308 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.324+0000 7feb8cb84700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb880ffe00 0x7feb8806a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.17:42682/0 (socket says 172.21.15.17:42682) 2023-07-14T12:01:01.308 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.324+0000 7feb8d385700 1 -- 172.21.15.17:0/3404921324 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb880ffe00 msgr2=0x7feb8806a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:01.308 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.324+0000 7feb8d385700 1 --2- 172.21.15.17:0/3404921324 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb880ffe00 0x7feb8806a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:01.308 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.324+0000 7feb8d385700 1 -- 172.21.15.17:0/3404921324 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb881007a0 msgr2=0x7feb8806ad80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T12:01:01.308 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.324+0000 7feb8d385700 1 --2- 172.21.15.17:0/3404921324 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb881007a0 0x7feb8806ad80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:01.309 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.324+0000 7feb8d385700 1 -- 172.21.15.17:0/3404921324 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb740095f0 con 0x7feb88104ea0 2023-07-14T12:01:01.309 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.325+0000 7feb8d385700 1 --2- 172.21.15.17:0/3404921324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb88104ea0 0x7feb8806b2c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7feb80007ee0 tx=0x7feb8000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:01.309 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.325+0000 7feb85ffb700 1 -- 172.21.15.17:0/3404921324 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb80039070 con 0x7feb88104ea0 2023-07-14T12:01:01.309 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.325+0000 7feb8ede8700 1 -- 172.21.15.17:0/3404921324 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb880732b0 con 0x7feb88104ea0 2023-07-14T12:01:01.309 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.325+0000 7feb85ffb700 1 -- 172.21.15.17:0/3404921324 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb80009390 con 0x7feb88104ea0 2023-07-14T12:01:01.309 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.325+0000 7feb85ffb700 1 -- 172.21.15.17:0/3404921324 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb80005860 con 0x7feb88104ea0 2023-07-14T12:01:01.309 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.325+0000 7feb8ede8700 1 -- 172.21.15.17:0/3404921324 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb880737b0 con 0x7feb88104ea0 2023-07-14T12:01:01.310 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.327+0000 7feb8ede8700 1 -- 172.21.15.17:0/3404921324 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb8804e210 con 0x7feb88104ea0 2023-07-14T12:01:01.311 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.329+0000 7feb85ffb700 1 -- 172.21.15.17:0/3404921324 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7feb80004fa0 con 0x7feb88104ea0 2023-07-14T12:01:01.312 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.331+0000 7feb85ffb700 1 --2- 172.21.15.17:0/3404921324 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7feb780a11d0 0x7feb780a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:01.312 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.331+0000 7feb85ffb700 1 -- 172.21.15.17:0/3404921324 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7feb800ee4b0 con 0x7feb88104ea0 2023-07-14T12:01:01.312 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.331+0000 7feb8cb84700 1 --2- 172.21.15.17:0/3404921324 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7feb780a11d0 0x7feb780a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:01.312 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.332+0000 7feb8cb84700 1 --2- 172.21.15.17:0/3404921324 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7feb780a11d0 0x7feb780a3680 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7feb7c009c60 tx=0x7feb7c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:01.315 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.334+0000 7feb85ffb700 1 -- 172.21.15.17:0/3404921324 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7feb800b78f0 con 0x7feb88104ea0 2023-07-14T12:01:01.470 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.490+0000 7f8ee2469700 1 -- 172.21.15.17:0/2463804784 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f8edc1a4ef0 con 0x7f8edc0ffe00 2023-07-14T12:01:01.471 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.491+0000 7f8ed97fa700 1 -- 172.21.15.17:0/2463804784 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f8ecc0b8e60 con 0x7f8edc0ffe00 2023-07-14T12:01:01.471 INFO:teuthology.orchestra.run.smithi017.stdout:339302416411 2023-07-14T12:01:01.473 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.493+0000 7f8ee2469700 1 -- 172.21.15.17:0/2463804784 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f8ec40a1150 msgr2=0x7f8ec40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:01.474 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.493+0000 7f8ee2469700 1 --2- 172.21.15.17:0/2463804784 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f8ec40a1150 0x7f8ec40a3600 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0009990 tx=0x7f8ed001a040 comp rx=0 tx=0).stop 2023-07-14T12:01:01.474 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.493+0000 7f8ee2469700 1 -- 172.21.15.17:0/2463804784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8edc0ffe00 msgr2=0x7f8edc197f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:01.474 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.493+0000 7f8ee2469700 1 --2- 172.21.15.17:0/2463804784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8edc0ffe00 0x7f8edc197f30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc00e9a0 tx=0x7f8ecc00ed60 comp rx=0 tx=0).stop 2023-07-14T12:01:01.474 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.493+0000 7f8ee2469700 1 -- 172.21.15.17:0/2463804784 shutdown_connections 2023-07-14T12:01:01.474 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.493+0000 7f8ee2469700 1 --2- 172.21.15.17:0/2463804784 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f8ec40a1150 0x7f8ec40a3600 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:01.474 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.493+0000 7f8ee2469700 1 --2- 172.21.15.17:0/2463804784 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8edc104ea0 0x7f8edc19c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:01.474 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.493+0000 7f8ee2469700 1 --2- 172.21.15.17:0/2463804784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8edc0ffe00 0x7f8edc197f30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:01.475 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.493+0000 7f8ee2469700 1 --2- 172.21.15.17:0/2463804784 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8edc1007a0 0x7f8edc198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:01.475 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.493+0000 7f8ee2469700 1 -- 172.21.15.17:0/2463804784 >> 172.21.15.17:0/2463804784 conn(0x7f8edc0fb660 msgr2=0x7f8edc0fc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:01.475 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.494+0000 7f8ee2469700 1 -- 172.21.15.17:0/2463804784 shutdown_connections 2023-07-14T12:01:01.475 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.494+0000 7f8ee2469700 1 -- 172.21.15.17:0/2463804784 wait complete. 2023-07-14T12:01:01.561 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.581+0000 7feb8ede8700 1 -- 172.21.15.17:0/3404921324 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7feb88073b30 con 0x7feb88104ea0 2023-07-14T12:01:01.562 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.582+0000 7feb85ffb700 1 -- 172.21.15.17:0/3404921324 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7feb800baf10 con 0x7feb88104ea0 2023-07-14T12:01:01.562 INFO:teuthology.orchestra.run.smithi017.stdout:309237645343 2023-07-14T12:01:01.565 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.584+0000 7feb8ede8700 1 -- 172.21.15.17:0/3404921324 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7feb780a11d0 msgr2=0x7feb780a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:01.565 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.584+0000 7feb8ede8700 1 --2- 172.21.15.17:0/3404921324 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7feb780a11d0 0x7feb780a3680 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7feb7c009c60 tx=0x7feb7c009260 comp rx=0 tx=0).stop 2023-07-14T12:01:01.565 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.584+0000 7feb8ede8700 1 -- 172.21.15.17:0/3404921324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb88104ea0 msgr2=0x7feb8806b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:01.566 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.584+0000 7feb8ede8700 1 --2- 172.21.15.17:0/3404921324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb88104ea0 0x7feb8806b2c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7feb80007ee0 tx=0x7feb8000d3b0 comp rx=0 tx=0).stop 2023-07-14T12:01:01.566 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.585+0000 7feb8ede8700 1 -- 172.21.15.17:0/3404921324 shutdown_connections 2023-07-14T12:01:01.566 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.585+0000 7feb8ede8700 1 --2- 172.21.15.17:0/3404921324 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7feb780a11d0 0x7feb780a3680 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:01.566 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.585+0000 7feb8ede8700 1 --2- 172.21.15.17:0/3404921324 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb880ffe00 0x7feb8806a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:01.566 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.585+0000 7feb8ede8700 1 --2- 172.21.15.17:0/3404921324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb88104ea0 0x7feb8806b2c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:01.566 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.585+0000 7feb8ede8700 1 --2- 172.21.15.17:0/3404921324 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb881007a0 0x7feb8806ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:01.566 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.585+0000 7feb8ede8700 1 -- 172.21.15.17:0/3404921324 >> 172.21.15.17:0/3404921324 conn(0x7feb880fb660 msgr2=0x7feb880fc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:01.566 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.585+0000 7feb8ede8700 1 -- 172.21.15.17:0/3404921324 shutdown_connections 2023-07-14T12:01:01.567 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.585+0000 7feb8ede8700 1 -- 172.21.15.17:0/3404921324 wait complete. 2023-07-14T12:01:02.015 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.912+0000 7fbed924a700 1 -- 172.21.15.17:0/3187361652 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbed41007a0 msgr2=0x7fbed4104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:02.016 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.912+0000 7fbed924a700 1 --2- 172.21.15.17:0/3187361652 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbed41007a0 0x7fbed4104770 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbec80099c0 tx=0x7fbec8009cd0 comp rx=0 tx=0).stop 2023-07-14T12:01:02.016 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.912+0000 7fbed924a700 1 -- 172.21.15.17:0/3187361652 shutdown_connections 2023-07-14T12:01:02.016 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.912+0000 7fbed924a700 1 --2- 172.21.15.17:0/3187361652 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbed4104ea0 0x7fbed410ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:02.016 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.912+0000 7fbed924a700 1 --2- 172.21.15.17:0/3187361652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbed40ffe00 0x7fbed41001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:02.016 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.912+0000 7fbed924a700 1 --2- 172.21.15.17:0/3187361652 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbed41007a0 0x7fbed4104770 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:02.016 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.912+0000 7fbed924a700 1 -- 172.21.15.17:0/3187361652 >> 172.21.15.17:0/3187361652 conn(0x7fbed40fb660 msgr2=0x7fbed40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:02.017 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.912+0000 7fbed924a700 1 -- 172.21.15.17:0/3187361652 shutdown_connections 2023-07-14T12:01:02.017 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.912+0000 7fbed924a700 1 -- 172.21.15.17:0/3187361652 wait complete. 2023-07-14T12:01:02.017 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.914+0000 7fbed924a700 1 Processor -- start 2023-07-14T12:01:02.017 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.914+0000 7fbed924a700 1 -- start start 2023-07-14T12:01:02.017 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.914+0000 7fbed924a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbed40ffe00 0x7fbed4195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:02.017 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.915+0000 7fbed924a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbed41007a0 0x7fbed4196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:02.017 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.915+0000 7fbed924a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbed4104ea0 0x7fbed419a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:02.018 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.915+0000 7fbed924a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed419acf0 con 0x7fbed4104ea0 2023-07-14T12:01:02.018 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.915+0000 7fbed924a700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed419ae60 con 0x7fbed40ffe00 2023-07-14T12:01:02.018 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.915+0000 7fbed924a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed419b020 con 0x7fbed41007a0 2023-07-14T12:01:02.018 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.915+0000 7fbed359e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbed4104ea0 0x7fbed419a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:02.018 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.915+0000 7fbed2d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbed40ffe00 0x7fbed4195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:02.018 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.915+0000 7fbed359e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbed4104ea0 0x7fbed419a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55092/0 (socket says 172.21.15.17:55092) 2023-07-14T12:01:02.018 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.915+0000 7fbed359e700 1 -- 172.21.15.17:0/1096905815 learned_addr learned my addr 172.21.15.17:0/1096905815 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:02.019 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.915+0000 7fbed259c700 1 --2- 172.21.15.17:0/1096905815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbed41007a0 0x7fbed4196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:02.019 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.915+0000 7fbed2d9d700 1 -- 172.21.15.17:0/1096905815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbed41007a0 msgr2=0x7fbed4196270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:02.019 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.915+0000 7fbed2d9d700 1 --2- 172.21.15.17:0/1096905815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbed41007a0 0x7fbed4196270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:02.019 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.915+0000 7fbed2d9d700 1 -- 172.21.15.17:0/1096905815 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbed4104ea0 msgr2=0x7fbed419a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:02.019 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.915+0000 7fbed2d9d700 1 --2- 172.21.15.17:0/1096905815 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbed4104ea0 0x7fbed419a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:02.019 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.915+0000 7fbed2d9d700 1 -- 172.21.15.17:0/1096905815 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec80095f0 con 0x7fbed40ffe00 2023-07-14T12:01:02.019 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.916+0000 7fbed2d9d700 1 --2- 172.21.15.17:0/1096905815 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbed40ffe00 0x7fbed4195d30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbec400e9f0 tx=0x7fbec400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:02.020 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.916+0000 7fbebbfff700 1 -- 172.21.15.17:0/1096905815 <== mon.2 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec400c9c0 con 0x7fbed40ffe00 2023-07-14T12:01:02.020 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.916+0000 7fbed924a700 1 -- 172.21.15.17:0/1096905815 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbed419b300 con 0x7fbed40ffe00 2023-07-14T12:01:02.020 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.916+0000 7fbebbfff700 1 -- 172.21.15.17:0/1096905815 <== mon.2 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbec4010430 con 0x7fbed40ffe00 2023-07-14T12:01:02.020 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.916+0000 7fbebbfff700 1 -- 172.21.15.17:0/1096905815 <== mon.2 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec4018680 con 0x7fbed40ffe00 2023-07-14T12:01:02.020 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.916+0000 7fbed924a700 1 -- 172.21.15.17:0/1096905815 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbed41a2b60 con 0x7fbed40ffe00 2023-07-14T12:01:02.020 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.917+0000 7fbed924a700 1 -- 172.21.15.17:0/1096905815 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbed404e210 con 0x7fbed40ffe00 2023-07-14T12:01:02.020 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.919+0000 7fbebbfff700 1 -- 172.21.15.17:0/1096905815 <== mon.2 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fbec4010860 con 0x7fbed40ffe00 2023-07-14T12:01:02.020 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.921+0000 7fbebbfff700 1 --2- 172.21.15.17:0/1096905815 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fbebc0a1150 0x7fbebc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:02.021 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.921+0000 7fbebbfff700 1 -- 172.21.15.17:0/1096905815 <== mon.2 v2:172.21.15.93:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fbec4014070 con 0x7fbed40ffe00 2023-07-14T12:01:02.021 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.921+0000 7fbed259c700 1 --2- 172.21.15.17:0/1096905815 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fbebc0a1150 0x7fbebc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:02.021 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.922+0000 7fbed259c700 1 --2- 172.21.15.17:0/1096905815 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fbebc0a1150 0x7fbebc0a3600 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbec8009fd0 tx=0x7fbec8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:02.021 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:01.925+0000 7fbebbfff700 1 -- 172.21.15.17:0/1096905815 <== mon.2 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fbec4051790 con 0x7fbed40ffe00 2023-07-14T12:01:02.081 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:01 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T12:01:02.081 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:01 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T12:01:02.081 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:01 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T12:01:02.081 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:01 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/2463804784' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-07-14T12:01:02.081 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:01 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/3404921324' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-07-14T12:01:02.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:01 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T12:01:02.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:01 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T12:01:02.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:01 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T12:01:02.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:01 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/2463804784' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-07-14T12:01:02.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:01 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/3404921324' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-07-14T12:01:02.123 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:02.143+0000 7fbed924a700 1 -- 172.21.15.17:0/1096905815 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fbed41a2cf0 con 0x7fbed40ffe00 2023-07-14T12:01:02.123 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:02.143+0000 7fbebbfff700 1 -- 172.21.15.17:0/1096905815 <== mon.2 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbec4096180 con 0x7fbed40ffe00 2023-07-14T12:01:02.124 INFO:teuthology.orchestra.run.smithi017.stdout:201863462961 2023-07-14T12:01:02.127 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:02.146+0000 7fbed924a700 1 -- 172.21.15.17:0/1096905815 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fbebc0a1150 msgr2=0x7fbebc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:02.127 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:02.146+0000 7fbed924a700 1 --2- 172.21.15.17:0/1096905815 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fbebc0a1150 0x7fbebc0a3600 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbec8009fd0 tx=0x7fbec8005fb0 comp rx=0 tx=0).stop 2023-07-14T12:01:02.127 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:02.146+0000 7fbed924a700 1 -- 172.21.15.17:0/1096905815 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbed40ffe00 msgr2=0x7fbed4195d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:02.127 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:02.146+0000 7fbed924a700 1 --2- 172.21.15.17:0/1096905815 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbed40ffe00 0x7fbed4195d30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbec400e9f0 tx=0x7fbec400ed00 comp rx=0 tx=0).stop 2023-07-14T12:01:02.127 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:02.146+0000 7fbed924a700 1 -- 172.21.15.17:0/1096905815 shutdown_connections 2023-07-14T12:01:02.127 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:02.146+0000 7fbed924a700 1 --2- 172.21.15.17:0/1096905815 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fbebc0a1150 0x7fbebc0a3600 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:02.128 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:02.146+0000 7fbed924a700 1 --2- 172.21.15.17:0/1096905815 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbed40ffe00 0x7fbed4195d30 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:02.128 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:02.146+0000 7fbed924a700 1 --2- 172.21.15.17:0/1096905815 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbed4104ea0 0x7fbed419a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:02.128 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:02.146+0000 7fbed924a700 1 --2- 172.21.15.17:0/1096905815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbed41007a0 0x7fbed4196270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:02.128 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:02.146+0000 7fbed924a700 1 -- 172.21.15.17:0/1096905815 >> 172.21.15.17:0/1096905815 conn(0x7fbed40fb660 msgr2=0x7fbed40fc170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:02.128 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:02.146+0000 7fbed924a700 1 -- 172.21.15.17:0/1096905815 shutdown_connections 2023-07-14T12:01:02.128 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:02.146+0000 7fbed924a700 1 -- 172.21.15.17:0/1096905815 wait complete. 2023-07-14T12:01:02.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:01 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T12:01:02.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:01 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T12:01:02.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:01 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' 2023-07-14T12:01:02.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:01 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/2463804784' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-07-14T12:01:02.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:01 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/3404921324' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-07-14T12:01:02.141 INFO:tasks.cephadm.ceph_manager.ceph:need seq 339302416408 got 339302416411 for osd.11 2023-07-14T12:01:02.141 DEBUG:teuthology.parallel:result is None 2023-07-14T12:01:02.840 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645340 got 309237645343 for osd.10 2023-07-14T12:01:02.841 DEBUG:teuthology.parallel:result is None 2023-07-14T12:01:02.934 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462958 got 201863462961 for osd.6 2023-07-14T12:01:02.934 DEBUG:teuthology.parallel:result is None 2023-07-14T12:01:02.935 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-07-14T12:01:02.935 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph pg dump --format=json 2023-07-14T12:01:03.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:02 smithi093 ceph-mon[128922]: pgmap v318: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-07-14T12:01:03.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:02 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/1096905815' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-07-14T12:01:03.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:02 smithi097 ceph-mon[129313]: pgmap v318: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-07-14T12:01:03.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:02 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/1096905815' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-07-14T12:01:03.143 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:02 smithi017 ceph-mon[126339]: pgmap v318: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-07-14T12:01:03.143 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:02 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/1096905815' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-07-14T12:01:03.670 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T12:01:04.610 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:04 smithi017 ceph-mon[126339]: pgmap v319: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-07-14T12:01:04.860 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:04 smithi093 ceph-mon[128922]: pgmap v319: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-07-14T12:01:04.884 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:04 smithi097 ceph-mon[129313]: pgmap v319: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-07-14T12:01:05.240 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.259+0000 7f10d50cf700 1 -- 172.21.15.17:0/3651767763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10d00ffe00 msgr2=0x7f10d01001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:05.241 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.259+0000 7f10d50cf700 1 --2- 172.21.15.17:0/3651767763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10d00ffe00 0x7f10d01001d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f10c00099c0 tx=0x7f10c0009cd0 comp rx=0 tx=0).stop 2023-07-14T12:01:05.241 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.259+0000 7f10d50cf700 1 -- 172.21.15.17:0/3651767763 shutdown_connections 2023-07-14T12:01:05.241 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.259+0000 7f10d50cf700 1 --2- 172.21.15.17:0/3651767763 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10d0104ea0 0x7f10d010ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:05.241 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.259+0000 7f10d50cf700 1 --2- 172.21.15.17:0/3651767763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10d00ffe00 0x7f10d01001d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:05.241 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.259+0000 7f10d50cf700 1 --2- 172.21.15.17:0/3651767763 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f10d01007a0 0x7f10d0104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:05.242 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.259+0000 7f10d50cf700 1 -- 172.21.15.17:0/3651767763 >> 172.21.15.17:0/3651767763 conn(0x7f10d00fb660 msgr2=0x7f10d00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:05.242 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.260+0000 7f10d50cf700 1 -- 172.21.15.17:0/3651767763 shutdown_connections 2023-07-14T12:01:05.242 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.260+0000 7f10d50cf700 1 -- 172.21.15.17:0/3651767763 wait complete. 2023-07-14T12:01:05.242 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.260+0000 7f10d50cf700 1 Processor -- start 2023-07-14T12:01:05.243 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.261+0000 7f10d50cf700 1 -- start start 2023-07-14T12:01:05.244 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.261+0000 7f10d50cf700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10d00ffe00 0x7f10d0197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:05.244 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.261+0000 7f10d50cf700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10d01007a0 0x7f10d01983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:05.244 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.261+0000 7f10d50cf700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f10d0104ea0 0x7f10d019c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:05.244 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.261+0000 7f10d50cf700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d019ce70 con 0x7f10d00ffe00 2023-07-14T12:01:05.244 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.261+0000 7f10d50cf700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d019cfe0 con 0x7f10d0104ea0 2023-07-14T12:01:05.244 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.261+0000 7f10d50cf700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d019d1a0 con 0x7f10d01007a0 2023-07-14T12:01:05.245 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.261+0000 7f10ced9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10d00ffe00 0x7f10d0197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:05.245 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.261+0000 7f10ced9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10d00ffe00 0x7f10d0197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49646/0 (socket says 172.21.15.17:49646) 2023-07-14T12:01:05.245 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.261+0000 7f10ced9d700 1 -- 172.21.15.17:0/2504020474 learned_addr learned my addr 172.21.15.17:0/2504020474 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:05.245 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.262+0000 7f10cf59e700 1 --2- 172.21.15.17:0/2504020474 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f10d0104ea0 0x7f10d019c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:05.245 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.262+0000 7f10ce59c700 1 --2- 172.21.15.17:0/2504020474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10d01007a0 0x7f10d01983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:05.245 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.262+0000 7f10ced9d700 1 -- 172.21.15.17:0/2504020474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10d01007a0 msgr2=0x7f10d01983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:05.246 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.262+0000 7f10ced9d700 1 --2- 172.21.15.17:0/2504020474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10d01007a0 0x7f10d01983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:05.246 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.262+0000 7f10ced9d700 1 -- 172.21.15.17:0/2504020474 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f10d0104ea0 msgr2=0x7f10d019c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:05.246 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.262+0000 7f10ced9d700 1 --2- 172.21.15.17:0/2504020474 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f10d0104ea0 0x7f10d019c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:05.246 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.262+0000 7f10ced9d700 1 -- 172.21.15.17:0/2504020474 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c8009c40 con 0x7f10d00ffe00 2023-07-14T12:01:05.246 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.262+0000 7f10ced9d700 1 --2- 172.21.15.17:0/2504020474 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10d00ffe00 0x7f10d0197eb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f10c00098e0 tx=0x7f10c000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:05.246 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.262+0000 7f10b7fff700 1 -- 172.21.15.17:0/2504020474 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c001d070 con 0x7f10d00ffe00 2023-07-14T12:01:05.246 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.263+0000 7f10d50cf700 1 -- 172.21.15.17:0/2504020474 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10c00095f0 con 0x7f10d00ffe00 2023-07-14T12:01:05.246 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.263+0000 7f10b7fff700 1 -- 172.21.15.17:0/2504020474 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10c000fb80 con 0x7f10d00ffe00 2023-07-14T12:01:05.247 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.263+0000 7f10b7fff700 1 -- 172.21.15.17:0/2504020474 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c0022df0 con 0x7f10d00ffe00 2023-07-14T12:01:05.247 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.263+0000 7f10d50cf700 1 -- 172.21.15.17:0/2504020474 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10d01a4b10 con 0x7f10d00ffe00 2023-07-14T12:01:05.248 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.264+0000 7f10d50cf700 1 -- 172.21.15.17:0/2504020474 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10d004e210 con 0x7f10d00ffe00 2023-07-14T12:01:05.248 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.266+0000 7f10b7fff700 1 -- 172.21.15.17:0/2504020474 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f10c000fcf0 con 0x7f10d00ffe00 2023-07-14T12:01:05.248 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.267+0000 7f10b7fff700 1 --2- 172.21.15.17:0/2504020474 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f10b80a11d0 0x7f10b80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:05.248 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.267+0000 7f10b7fff700 1 -- 172.21.15.17:0/2504020474 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f10c00cdc50 con 0x7f10d00ffe00 2023-07-14T12:01:05.248 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.267+0000 7f10ce59c700 1 --2- 172.21.15.17:0/2504020474 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f10b80a11d0 0x7f10b80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:05.249 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.268+0000 7f10ce59c700 1 --2- 172.21.15.17:0/2504020474 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f10b80a11d0 0x7f10b80a3680 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f10d01994d0 tx=0x7f10c4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:05.251 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.271+0000 7f10b7fff700 1 -- 172.21.15.17:0/2504020474 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f10c00970c0 con 0x7f10d00ffe00 2023-07-14T12:01:05.450 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.470+0000 7f10d50cf700 1 -- 172.21.15.17:0/2504020474 --> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f10d010c400 con 0x7f10b80a11d0 2023-07-14T12:01:05.464 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.484+0000 7f10b7fff700 1 -- 172.21.15.17:0/2504020474 <== mgr.14152 v2:172.21.15.17:6800/4237449858 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427598 (secure 0 0 0) 0x7f10d010c400 con 0x7f10b80a11d0 2023-07-14T12:01:05.471 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T12:01:05.473 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.491+0000 7f10d50cf700 1 -- 172.21.15.17:0/2504020474 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f10b80a11d0 msgr2=0x7f10b80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:05.473 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.491+0000 7f10d50cf700 1 --2- 172.21.15.17:0/2504020474 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f10b80a11d0 0x7f10b80a3680 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f10d01994d0 tx=0x7f10c4009260 comp rx=0 tx=0).stop 2023-07-14T12:01:05.474 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.491+0000 7f10d50cf700 1 -- 172.21.15.17:0/2504020474 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10d00ffe00 msgr2=0x7f10d0197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:05.474 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.491+0000 7f10d50cf700 1 --2- 172.21.15.17:0/2504020474 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10d00ffe00 0x7f10d0197eb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f10c00098e0 tx=0x7f10c000f690 comp rx=0 tx=0).stop 2023-07-14T12:01:05.474 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.491+0000 7f10d50cf700 1 -- 172.21.15.17:0/2504020474 shutdown_connections 2023-07-14T12:01:05.474 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.491+0000 7f10d50cf700 1 --2- 172.21.15.17:0/2504020474 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f10b80a11d0 0x7f10b80a3680 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:05.474 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.491+0000 7f10d50cf700 1 --2- 172.21.15.17:0/2504020474 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f10d0104ea0 0x7f10d019c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:05.474 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.491+0000 7f10d50cf700 1 --2- 172.21.15.17:0/2504020474 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10d00ffe00 0x7f10d0197eb0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:05.474 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.491+0000 7f10d50cf700 1 --2- 172.21.15.17:0/2504020474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10d01007a0 0x7f10d01983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:05.475 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.491+0000 7f10d50cf700 1 -- 172.21.15.17:0/2504020474 >> 172.21.15.17:0/2504020474 conn(0x7f10d00fb660 msgr2=0x7f10d010bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:05.475 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.491+0000 7f10d50cf700 1 -- 172.21.15.17:0/2504020474 shutdown_connections 2023-07-14T12:01:05.475 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:05.491+0000 7f10d50cf700 1 -- 172.21.15.17:0/2504020474 wait complete. 2023-07-14T12:01:05.475 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-07-14T12:01:05.969 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":319,"stamp":"2023-07-14T12:01:03.746727+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":600392,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":624,"num_read_kb":385,"num_write":390,"num_write_kb":3005,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":320,"ondisk_log_size":320,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335316,"kb_used_data":11892,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606508,"statfs":{"total":1151940427776,"available":1151597064192,"internally_reserved":0,"allocated":12177408,"data_stored":7634763,"data_compressed":319254,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":140,"num_read_kb":75,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003506"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.813006+0000","last_change":"2023-07-14T11:59:43.468306+0000","last_active":"2023-07-14T12:00:06.813006+0000","last_peered":"2023-07-14T12:00:06.813006+0000","last_clean":"2023-07-14T12:00:06.813006+0000","last_became_active":"2023-07-14T11:59:43.467506+0000","last_became_peered":"2023-07-14T11:59:43.467506+0000","last_unstale":"2023-07-14T12:00:06.813006+0000","last_undegraded":"2023-07-14T12:00:06.813006+0000","last_fullsized":"2023-07-14T12:00:06.813006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:49:26.343044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.814808+0000","last_change":"2023-07-14T11:59:43.462562+0000","last_active":"2023-07-14T12:00:06.814808+0000","last_peered":"2023-07-14T12:00:06.814808+0000","last_clean":"2023-07-14T12:00:06.814808+0000","last_became_active":"2023-07-14T11:59:43.462126+0000","last_became_peered":"2023-07-14T11:59:43.462126+0000","last_unstale":"2023-07-14T12:00:06.814808+0000","last_undegraded":"2023-07-14T12:00:06.814808+0000","last_fullsized":"2023-07-14T12:00:06.814808+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:20:10.445422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.813957+0000","last_change":"2023-07-14T11:59:43.470205+0000","last_active":"2023-07-14T12:00:06.813957+0000","last_peered":"2023-07-14T12:00:06.813957+0000","last_clean":"2023-07-14T12:00:06.813957+0000","last_became_active":"2023-07-14T11:59:43.469804+0000","last_became_peered":"2023-07-14T11:59:43.469804+0000","last_unstale":"2023-07-14T12:00:06.813957+0000","last_undegraded":"2023-07-14T12:00:06.813957+0000","last_fullsized":"2023-07-14T12:00:06.813957+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:06:21.532674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.807812+0000","last_change":"2023-07-14T11:59:43.464834+0000","last_active":"2023-07-14T12:00:06.807812+0000","last_peered":"2023-07-14T12:00:06.807812+0000","last_clean":"2023-07-14T12:00:06.807812+0000","last_became_active":"2023-07-14T11:59:43.464252+0000","last_became_peered":"2023-07-14T11:59:43.464252+0000","last_unstale":"2023-07-14T12:00:06.807812+0000","last_undegraded":"2023-07-14T12:00:06.807812+0000","last_fullsized":"2023-07-14T12:00:06.807812+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:30:26.528049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808013+0000","last_change":"2023-07-14T11:59:43.460468+0000","last_active":"2023-07-14T12:00:06.808013+0000","last_peered":"2023-07-14T12:00:06.808013+0000","last_clean":"2023-07-14T12:00:06.808013+0000","last_became_active":"2023-07-14T11:59:43.460050+0000","last_became_peered":"2023-07-14T11:59:43.460050+0000","last_unstale":"2023-07-14T12:00:06.808013+0000","last_undegraded":"2023-07-14T12:00:06.808013+0000","last_fullsized":"2023-07-14T12:00:06.808013+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:43:21.915056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808330+0000","last_change":"2023-07-14T11:59:43.458605+0000","last_active":"2023-07-14T12:00:06.808330+0000","last_peered":"2023-07-14T12:00:06.808330+0000","last_clean":"2023-07-14T12:00:06.808330+0000","last_became_active":"2023-07-14T11:59:43.458233+0000","last_became_peered":"2023-07-14T11:59:43.458233+0000","last_unstale":"2023-07-14T12:00:06.808330+0000","last_undegraded":"2023-07-14T12:00:06.808330+0000","last_fullsized":"2023-07-14T12:00:06.808330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:27:55.968254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.328508+0000","last_change":"2023-07-14T11:59:43.465950+0000","last_active":"2023-07-14T12:00:07.328508+0000","last_peered":"2023-07-14T12:00:07.328508+0000","last_clean":"2023-07-14T12:00:07.328508+0000","last_became_active":"2023-07-14T11:59:43.465652+0000","last_became_peered":"2023-07-14T11:59:43.465652+0000","last_unstale":"2023-07-14T12:00:07.328508+0000","last_undegraded":"2023-07-14T12:00:07.328508+0000","last_fullsized":"2023-07-14T12:00:07.328508+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:48:42.466464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.816094+0000","last_change":"2023-07-14T12:00:06.812609+0000","last_active":"2023-07-14T12:00:06.816094+0000","last_peered":"2023-07-14T12:00:06.816094+0000","last_clean":"2023-07-14T12:00:06.816094+0000","last_became_active":"2023-07-14T12:00:06.812112+0000","last_became_peered":"2023-07-14T12:00:06.812112+0000","last_unstale":"2023-07-14T12:00:06.816094+0000","last_undegraded":"2023-07-14T12:00:06.816094+0000","last_fullsized":"2023-07-14T12:00:06.816094+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:41:31.808111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805908+0000","last_change":"2023-07-14T11:59:43.466351+0000","last_active":"2023-07-14T12:00:06.805908+0000","last_peered":"2023-07-14T12:00:06.805908+0000","last_clean":"2023-07-14T12:00:06.805908+0000","last_became_active":"2023-07-14T11:59:43.465442+0000","last_became_peered":"2023-07-14T11:59:43.465442+0000","last_unstale":"2023-07-14T12:00:06.805908+0000","last_undegraded":"2023-07-14T12:00:06.805908+0000","last_fullsized":"2023-07-14T12:00:06.805908+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:52:04.297945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.809112+0000","last_change":"2023-07-14T11:59:43.457684+0000","last_active":"2023-07-14T12:00:06.809112+0000","last_peered":"2023-07-14T12:00:06.809112+0000","last_clean":"2023-07-14T12:00:06.809112+0000","last_became_active":"2023-07-14T11:59:43.457442+0000","last_became_peered":"2023-07-14T11:59:43.457442+0000","last_unstale":"2023-07-14T12:00:06.809112+0000","last_undegraded":"2023-07-14T12:00:06.809112+0000","last_fullsized":"2023-07-14T12:00:06.809112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:06:41.592243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.328566+0000","last_change":"2023-07-14T11:59:43.463652+0000","last_active":"2023-07-14T12:00:07.328566+0000","last_peered":"2023-07-14T12:00:07.328566+0000","last_clean":"2023-07-14T12:00:07.328566+0000","last_became_active":"2023-07-14T11:59:43.463311+0000","last_became_peered":"2023-07-14T11:59:43.463311+0000","last_unstale":"2023-07-14T12:00:07.328566+0000","last_undegraded":"2023-07-14T12:00:07.328566+0000","last_fullsized":"2023-07-14T12:00:07.328566+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:28:35.177490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808344+0000","last_change":"2023-07-14T11:59:43.464633+0000","last_active":"2023-07-14T12:00:06.808344+0000","last_peered":"2023-07-14T12:00:06.808344+0000","last_clean":"2023-07-14T12:00:06.808344+0000","last_became_active":"2023-07-14T11:59:43.463847+0000","last_became_peered":"2023-07-14T11:59:43.463847+0000","last_unstale":"2023-07-14T12:00:06.808344+0000","last_undegraded":"2023-07-14T12:00:06.808344+0000","last_fullsized":"2023-07-14T12:00:06.808344+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:08:28.236596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808316+0000","last_change":"2023-07-14T11:59:43.465650+0000","last_active":"2023-07-14T12:00:06.808316+0000","last_peered":"2023-07-14T12:00:06.808316+0000","last_clean":"2023-07-14T12:00:06.808316+0000","last_became_active":"2023-07-14T11:59:43.465276+0000","last_became_peered":"2023-07-14T11:59:43.465276+0000","last_unstale":"2023-07-14T12:00:06.808316+0000","last_undegraded":"2023-07-14T12:00:06.808316+0000","last_fullsized":"2023-07-14T12:00:06.808316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:00:25.094505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.812937+0000","last_change":"2023-07-14T11:59:43.466437+0000","last_active":"2023-07-14T12:00:06.812937+0000","last_peered":"2023-07-14T12:00:06.812937+0000","last_clean":"2023-07-14T12:00:06.812937+0000","last_became_active":"2023-07-14T11:59:43.465361+0000","last_became_peered":"2023-07-14T11:59:43.465361+0000","last_unstale":"2023-07-14T12:00:06.812937+0000","last_undegraded":"2023-07-14T12:00:06.812937+0000","last_fullsized":"2023-07-14T12:00:06.812937+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:31:26.002175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.814761+0000","last_change":"2023-07-14T11:59:43.469452+0000","last_active":"2023-07-14T12:00:06.814761+0000","last_peered":"2023-07-14T12:00:06.814761+0000","last_clean":"2023-07-14T12:00:06.814761+0000","last_became_active":"2023-07-14T11:59:43.469112+0000","last_became_peered":"2023-07-14T11:59:43.469112+0000","last_unstale":"2023-07-14T12:00:06.814761+0000","last_undegraded":"2023-07-14T12:00:06.814761+0000","last_fullsized":"2023-07-14T12:00:06.814761+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:25:31.596988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.804882+0000","last_change":"2023-07-14T11:59:43.466746+0000","last_active":"2023-07-14T12:00:06.804882+0000","last_peered":"2023-07-14T12:00:06.804882+0000","last_clean":"2023-07-14T12:00:06.804882+0000","last_became_active":"2023-07-14T11:59:43.466530+0000","last_became_peered":"2023-07-14T11:59:43.466530+0000","last_unstale":"2023-07-14T12:00:06.804882+0000","last_undegraded":"2023-07-14T12:00:06.804882+0000","last_fullsized":"2023-07-14T12:00:06.804882+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:47:15.633490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.812795+0000","last_change":"2023-07-14T11:59:43.468510+0000","last_active":"2023-07-14T12:00:06.812795+0000","last_peered":"2023-07-14T12:00:06.812795+0000","last_clean":"2023-07-14T12:00:06.812795+0000","last_became_active":"2023-07-14T11:59:43.467990+0000","last_became_peered":"2023-07-14T11:59:43.467990+0000","last_unstale":"2023-07-14T12:00:06.812795+0000","last_undegraded":"2023-07-14T12:00:06.812795+0000","last_fullsized":"2023-07-14T12:00:06.812795+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:35:21.496112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805431+0000","last_change":"2023-07-14T11:59:43.465290+0000","last_active":"2023-07-14T12:00:06.805431+0000","last_peered":"2023-07-14T12:00:06.805431+0000","last_clean":"2023-07-14T12:00:06.805431+0000","last_became_active":"2023-07-14T11:59:43.464982+0000","last_became_peered":"2023-07-14T11:59:43.464982+0000","last_unstale":"2023-07-14T12:00:06.805431+0000","last_undegraded":"2023-07-14T12:00:06.805431+0000","last_fullsized":"2023-07-14T12:00:06.805431+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:49:36.157904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.806693+0000","last_change":"2023-07-14T11:59:43.465053+0000","last_active":"2023-07-14T12:00:06.806693+0000","last_peered":"2023-07-14T12:00:06.806693+0000","last_clean":"2023-07-14T12:00:06.806693+0000","last_became_active":"2023-07-14T11:59:43.464759+0000","last_became_peered":"2023-07-14T11:59:43.464759+0000","last_unstale":"2023-07-14T12:00:06.806693+0000","last_undegraded":"2023-07-14T12:00:06.806693+0000","last_fullsized":"2023-07-14T12:00:06.806693+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:41:15.313602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805157+0000","last_change":"2023-07-14T11:59:47.792164+0000","last_active":"2023-07-14T12:00:06.805157+0000","last_peered":"2023-07-14T12:00:06.805157+0000","last_clean":"2023-07-14T12:00:06.805157+0000","last_became_active":"2023-07-14T11:59:47.791749+0000","last_became_peered":"2023-07-14T11:59:47.791749+0000","last_unstale":"2023-07-14T12:00:06.805157+0000","last_undegraded":"2023-07-14T12:00:06.805157+0000","last_fullsized":"2023-07-14T12:00:06.805157+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:32:07.262881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'13","reported_seq":85,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:01:00.561036+0000","last_change":"2023-07-14T11:59:42.759256+0000","last_active":"2023-07-14T12:01:00.561036+0000","last_peered":"2023-07-14T12:01:00.561036+0000","last_clean":"2023-07-14T12:01:00.561036+0000","last_became_active":"2023-07-14T11:59:42.758938+0000","last_became_peered":"2023-07-14T11:59:42.758938+0000","last_unstale":"2023-07-14T12:01:00.561036+0000","last_undegraded":"2023-07-14T12:01:00.561036+0000","last_fullsized":"2023-07-14T12:01:00.561036+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:21:38.079963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":46,"num_read_kb":46,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.811011+0000","last_change":"2023-07-14T11:59:43.467503+0000","last_active":"2023-07-14T12:00:06.811011+0000","last_peered":"2023-07-14T12:00:06.811011+0000","last_clean":"2023-07-14T12:00:06.811011+0000","last_became_active":"2023-07-14T11:59:43.467130+0000","last_became_peered":"2023-07-14T11:59:43.467130+0000","last_unstale":"2023-07-14T12:00:06.811011+0000","last_undegraded":"2023-07-14T12:00:06.811011+0000","last_fullsized":"2023-07-14T12:00:06.811011+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:26:08.037607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.807340+0000","last_change":"2023-07-14T11:59:47.783514+0000","last_active":"2023-07-14T12:00:06.807340+0000","last_peered":"2023-07-14T12:00:06.807340+0000","last_clean":"2023-07-14T12:00:06.807340+0000","last_became_active":"2023-07-14T11:59:47.783115+0000","last_became_peered":"2023-07-14T11:59:47.783115+0000","last_unstale":"2023-07-14T12:00:06.807340+0000","last_undegraded":"2023-07-14T12:00:06.807340+0000","last_fullsized":"2023-07-14T12:00:06.807340+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:19:30.538776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.811193+0000","last_change":"2023-07-14T11:59:42.768975+0000","last_active":"2023-07-14T12:00:06.811193+0000","last_peered":"2023-07-14T12:00:06.811193+0000","last_clean":"2023-07-14T12:00:06.811193+0000","last_became_active":"2023-07-14T11:59:42.768603+0000","last_became_peered":"2023-07-14T11:59:42.768603+0000","last_unstale":"2023-07-14T12:00:06.811193+0000","last_undegraded":"2023-07-14T12:00:06.811193+0000","last_fullsized":"2023-07-14T12:00:06.811193+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:20:32.308882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.813232+0000","last_change":"2023-07-14T11:59:43.474252+0000","last_active":"2023-07-14T12:00:06.813232+0000","last_peered":"2023-07-14T12:00:06.813232+0000","last_clean":"2023-07-14T12:00:06.813232+0000","last_became_active":"2023-07-14T11:59:43.473955+0000","last_became_peered":"2023-07-14T11:59:43.473955+0000","last_unstale":"2023-07-14T12:00:06.813232+0000","last_undegraded":"2023-07-14T12:00:06.813232+0000","last_fullsized":"2023-07-14T12:00:06.813232+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:52:46.217372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.813893+0000","last_change":"2023-07-14T12:00:06.813893+0000","last_active":"2023-07-14T12:00:06.813893+0000","last_peered":"2023-07-14T12:00:06.813893+0000","last_clean":"2023-07-14T12:00:06.813893+0000","last_became_active":"2023-07-14T12:00:06.813568+0000","last_became_peered":"2023-07-14T12:00:06.813568+0000","last_unstale":"2023-07-14T12:00:06.813893+0000","last_undegraded":"2023-07-14T12:00:06.813893+0000","last_fullsized":"2023-07-14T12:00:06.813893+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:57:58.591410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:17.499054+0000","last_change":"2023-07-14T11:59:42.760791+0000","last_active":"2023-07-14T12:00:17.499054+0000","last_peered":"2023-07-14T12:00:17.499054+0000","last_clean":"2023-07-14T12:00:17.499054+0000","last_became_active":"2023-07-14T11:59:42.760422+0000","last_became_peered":"2023-07-14T11:59:42.760422+0000","last_unstale":"2023-07-14T12:00:17.499054+0000","last_undegraded":"2023-07-14T12:00:17.499054+0000","last_fullsized":"2023-07-14T12:00:17.499054+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:10:21.008350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805235+0000","last_change":"2023-07-14T11:59:43.455829+0000","last_active":"2023-07-14T12:00:06.805235+0000","last_peered":"2023-07-14T12:00:06.805235+0000","last_clean":"2023-07-14T12:00:06.805235+0000","last_became_active":"2023-07-14T11:59:43.455541+0000","last_became_peered":"2023-07-14T11:59:43.455541+0000","last_unstale":"2023-07-14T12:00:06.805235+0000","last_undegraded":"2023-07-14T12:00:06.805235+0000","last_fullsized":"2023-07-14T12:00:06.805235+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:42:43.607658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.807537+0000","last_change":"2023-07-14T11:59:47.783496+0000","last_active":"2023-07-14T12:00:06.807537+0000","last_peered":"2023-07-14T12:00:06.807537+0000","last_clean":"2023-07-14T12:00:06.807537+0000","last_became_active":"2023-07-14T11:59:47.783128+0000","last_became_peered":"2023-07-14T11:59:47.783128+0000","last_unstale":"2023-07-14T12:00:06.807537+0000","last_undegraded":"2023-07-14T12:00:06.807537+0000","last_fullsized":"2023-07-14T12:00:06.807537+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:55:13.905682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:16.495600+0000","last_change":"2023-07-14T11:59:42.757255+0000","last_active":"2023-07-14T12:00:16.495600+0000","last_peered":"2023-07-14T12:00:16.495600+0000","last_clean":"2023-07-14T12:00:16.495600+0000","last_became_active":"2023-07-14T11:59:42.756882+0000","last_became_peered":"2023-07-14T11:59:42.756882+0000","last_unstale":"2023-07-14T12:00:16.495600+0000","last_undegraded":"2023-07-14T12:00:16.495600+0000","last_fullsized":"2023-07-14T12:00:16.495600+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:22:50.954980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808779+0000","last_change":"2023-07-14T11:59:43.465501+0000","last_active":"2023-07-14T12:00:06.808779+0000","last_peered":"2023-07-14T12:00:06.808779+0000","last_clean":"2023-07-14T12:00:06.808779+0000","last_became_active":"2023-07-14T11:59:43.465218+0000","last_became_peered":"2023-07-14T11:59:43.465218+0000","last_unstale":"2023-07-14T12:00:06.808779+0000","last_undegraded":"2023-07-14T12:00:06.808779+0000","last_fullsized":"2023-07-14T12:00:06.808779+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:02:45.983300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.806670+0000","last_change":"2023-07-14T11:59:47.777816+0000","last_active":"2023-07-14T12:00:06.806670+0000","last_peered":"2023-07-14T12:00:06.806670+0000","last_clean":"2023-07-14T12:00:06.806670+0000","last_became_active":"2023-07-14T11:59:47.777373+0000","last_became_peered":"2023-07-14T11:59:47.777373+0000","last_unstale":"2023-07-14T12:00:06.806670+0000","last_undegraded":"2023-07-14T12:00:06.806670+0000","last_fullsized":"2023-07-14T12:00:06.806670+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:58:17.252326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":76,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:58.559215+0000","last_change":"2023-07-14T11:59:42.756362+0000","last_active":"2023-07-14T12:00:58.559215+0000","last_peered":"2023-07-14T12:00:58.559215+0000","last_clean":"2023-07-14T12:00:58.559215+0000","last_became_active":"2023-07-14T11:59:42.756074+0000","last_became_peered":"2023-07-14T11:59:42.756074+0000","last_unstale":"2023-07-14T12:00:58.559215+0000","last_undegraded":"2023-07-14T12:00:58.559215+0000","last_fullsized":"2023-07-14T12:00:58.559215+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:22:56.214797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.812341+0000","last_change":"2023-07-14T12:00:06.812341+0000","last_active":"2023-07-14T12:00:06.812341+0000","last_peered":"2023-07-14T12:00:06.812341+0000","last_clean":"2023-07-14T12:00:06.812341+0000","last_became_active":"2023-07-14T12:00:06.811891+0000","last_became_peered":"2023-07-14T12:00:06.811891+0000","last_unstale":"2023-07-14T12:00:06.812341+0000","last_undegraded":"2023-07-14T12:00:06.812341+0000","last_fullsized":"2023-07-14T12:00:06.812341+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:19:54.770200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.806081+0000","last_change":"2023-07-14T11:59:47.785323+0000","last_active":"2023-07-14T12:00:06.806081+0000","last_peered":"2023-07-14T12:00:06.806081+0000","last_clean":"2023-07-14T12:00:06.806081+0000","last_became_active":"2023-07-14T11:59:47.784892+0000","last_became_peered":"2023-07-14T11:59:47.784892+0000","last_unstale":"2023-07-14T12:00:06.806081+0000","last_undegraded":"2023-07-14T12:00:06.806081+0000","last_fullsized":"2023-07-14T12:00:06.806081+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:48:54.236864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:15.490306+0000","last_change":"2023-07-14T11:59:42.756364+0000","last_active":"2023-07-14T12:00:15.490306+0000","last_peered":"2023-07-14T12:00:15.490306+0000","last_clean":"2023-07-14T12:00:15.490306+0000","last_became_active":"2023-07-14T11:59:42.756080+0000","last_became_peered":"2023-07-14T11:59:42.756080+0000","last_unstale":"2023-07-14T12:00:15.490306+0000","last_undegraded":"2023-07-14T12:00:15.490306+0000","last_fullsized":"2023-07-14T12:00:15.490306+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:55:28.974638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808819+0000","last_change":"2023-07-14T11:59:43.470331+0000","last_active":"2023-07-14T12:00:06.808819+0000","last_peered":"2023-07-14T12:00:06.808819+0000","last_clean":"2023-07-14T12:00:06.808819+0000","last_became_active":"2023-07-14T11:59:43.469806+0000","last_became_peered":"2023-07-14T11:59:43.469806+0000","last_unstale":"2023-07-14T12:00:06.808819+0000","last_undegraded":"2023-07-14T12:00:06.808819+0000","last_fullsized":"2023-07-14T12:00:06.808819+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:31:39.648120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.812748+0000","last_change":"2023-07-14T11:59:47.784083+0000","last_active":"2023-07-14T12:00:06.812748+0000","last_peered":"2023-07-14T12:00:06.812748+0000","last_clean":"2023-07-14T12:00:06.812748+0000","last_became_active":"2023-07-14T11:59:47.783624+0000","last_became_peered":"2023-07-14T11:59:47.783624+0000","last_unstale":"2023-07-14T12:00:06.812748+0000","last_undegraded":"2023-07-14T12:00:06.812748+0000","last_fullsized":"2023-07-14T12:00:06.812748+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:19:59.493479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:14.477781+0000","last_change":"2023-07-14T11:59:42.764869+0000","last_active":"2023-07-14T12:00:14.477781+0000","last_peered":"2023-07-14T12:00:14.477781+0000","last_clean":"2023-07-14T12:00:14.477781+0000","last_became_active":"2023-07-14T11:59:42.764514+0000","last_became_peered":"2023-07-14T11:59:42.764514+0000","last_unstale":"2023-07-14T12:00:14.477781+0000","last_undegraded":"2023-07-14T12:00:14.477781+0000","last_fullsized":"2023-07-14T12:00:14.477781+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:14:29.041985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.809908+0000","last_change":"2023-07-14T11:59:43.458165+0000","last_active":"2023-07-14T12:00:06.809908+0000","last_peered":"2023-07-14T12:00:06.809908+0000","last_clean":"2023-07-14T12:00:06.809908+0000","last_became_active":"2023-07-14T11:59:43.457829+0000","last_became_peered":"2023-07-14T11:59:43.457829+0000","last_unstale":"2023-07-14T12:00:06.809908+0000","last_undegraded":"2023-07-14T12:00:06.809908+0000","last_fullsized":"2023-07-14T12:00:06.809908+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:43:38.283487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.814633+0000","last_change":"2023-07-14T11:59:47.782676+0000","last_active":"2023-07-14T12:00:06.814633+0000","last_peered":"2023-07-14T12:00:06.814633+0000","last_clean":"2023-07-14T12:00:06.814633+0000","last_became_active":"2023-07-14T11:59:47.782283+0000","last_became_peered":"2023-07-14T11:59:47.782283+0000","last_unstale":"2023-07-14T12:00:06.814633+0000","last_undegraded":"2023-07-14T12:00:06.814633+0000","last_fullsized":"2023-07-14T12:00:06.814633+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:50:29.299244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:16.494916+0000","last_change":"2023-07-14T11:59:42.756701+0000","last_active":"2023-07-14T12:00:16.494916+0000","last_peered":"2023-07-14T12:00:16.494916+0000","last_clean":"2023-07-14T12:00:16.494916+0000","last_became_active":"2023-07-14T11:59:42.756452+0000","last_became_peered":"2023-07-14T11:59:42.756452+0000","last_unstale":"2023-07-14T12:00:16.494916+0000","last_undegraded":"2023-07-14T12:00:16.494916+0000","last_fullsized":"2023-07-14T12:00:16.494916+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:56:37.979449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.804650+0000","last_change":"2023-07-14T11:59:43.457029+0000","last_active":"2023-07-14T12:00:06.804650+0000","last_peered":"2023-07-14T12:00:06.804650+0000","last_clean":"2023-07-14T12:00:06.804650+0000","last_became_active":"2023-07-14T11:59:43.456710+0000","last_became_peered":"2023-07-14T11:59:43.456710+0000","last_unstale":"2023-07-14T12:00:06.804650+0000","last_undegraded":"2023-07-14T12:00:06.804650+0000","last_fullsized":"2023-07-14T12:00:06.804650+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:17:39.969654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.811649+0000","last_change":"2023-07-14T11:59:47.785732+0000","last_active":"2023-07-14T12:00:06.811649+0000","last_peered":"2023-07-14T12:00:06.811649+0000","last_clean":"2023-07-14T12:00:06.811649+0000","last_became_active":"2023-07-14T11:59:47.785289+0000","last_became_peered":"2023-07-14T11:59:47.785289+0000","last_unstale":"2023-07-14T12:00:06.811649+0000","last_undegraded":"2023-07-14T12:00:06.811649+0000","last_fullsized":"2023-07-14T12:00:06.811649+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:58:47.873312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":124,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:01:03.565696+0000","last_change":"2023-07-14T11:59:42.760013+0000","last_active":"2023-07-14T12:01:03.565696+0000","last_peered":"2023-07-14T12:01:03.565696+0000","last_clean":"2023-07-14T12:01:03.565696+0000","last_became_active":"2023-07-14T11:59:42.759673+0000","last_became_peered":"2023-07-14T11:59:42.759673+0000","last_unstale":"2023-07-14T12:01:03.565696+0000","last_undegraded":"2023-07-14T12:01:03.565696+0000","last_fullsized":"2023-07-14T12:01:03.565696+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:31:23.656642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":94,"num_read_kb":94,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805922+0000","last_change":"2023-07-14T11:59:43.455418+0000","last_active":"2023-07-14T12:00:06.805922+0000","last_peered":"2023-07-14T12:00:06.805922+0000","last_clean":"2023-07-14T12:00:06.805922+0000","last_became_active":"2023-07-14T11:59:43.454963+0000","last_became_peered":"2023-07-14T11:59:43.454963+0000","last_unstale":"2023-07-14T12:00:06.805922+0000","last_undegraded":"2023-07-14T12:00:06.805922+0000","last_fullsized":"2023-07-14T12:00:06.805922+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:35:27.262942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805369+0000","last_change":"2023-07-14T11:59:47.784330+0000","last_active":"2023-07-14T12:00:06.805369+0000","last_peered":"2023-07-14T12:00:06.805369+0000","last_clean":"2023-07-14T12:00:06.805369+0000","last_became_active":"2023-07-14T11:59:47.783640+0000","last_became_peered":"2023-07-14T11:59:47.783640+0000","last_unstale":"2023-07-14T12:00:06.805369+0000","last_undegraded":"2023-07-14T12:00:06.805369+0000","last_fullsized":"2023-07-14T12:00:06.805369+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:23:46.613595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:16.496542+0000","last_change":"2023-07-14T11:59:42.758856+0000","last_active":"2023-07-14T12:00:16.496542+0000","last_peered":"2023-07-14T12:00:16.496542+0000","last_clean":"2023-07-14T12:00:16.496542+0000","last_became_active":"2023-07-14T11:59:42.758571+0000","last_became_peered":"2023-07-14T11:59:42.758571+0000","last_unstale":"2023-07-14T12:00:16.496542+0000","last_undegraded":"2023-07-14T12:00:16.496542+0000","last_fullsized":"2023-07-14T12:00:16.496542+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:22:26.607862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.814254+0000","last_change":"2023-07-14T11:59:43.466302+0000","last_active":"2023-07-14T12:00:06.814254+0000","last_peered":"2023-07-14T12:00:06.814254+0000","last_clean":"2023-07-14T12:00:06.814254+0000","last_became_active":"2023-07-14T11:59:43.465958+0000","last_became_peered":"2023-07-14T11:59:43.465958+0000","last_unstale":"2023-07-14T12:00:06.814254+0000","last_undegraded":"2023-07-14T12:00:06.814254+0000","last_fullsized":"2023-07-14T12:00:06.814254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:09:00.018902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.813066+0000","last_change":"2023-07-14T12:00:06.813066+0000","last_active":"2023-07-14T12:00:06.813066+0000","last_peered":"2023-07-14T12:00:06.813066+0000","last_clean":"2023-07-14T12:00:06.813066+0000","last_became_active":"2023-07-14T12:00:06.812460+0000","last_became_peered":"2023-07-14T12:00:06.812460+0000","last_unstale":"2023-07-14T12:00:06.813066+0000","last_undegraded":"2023-07-14T12:00:06.813066+0000","last_fullsized":"2023-07-14T12:00:06.813066+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:47:40.893162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:15.487525+0000","last_change":"2023-07-14T11:59:42.765777+0000","last_active":"2023-07-14T12:00:15.487525+0000","last_peered":"2023-07-14T12:00:15.487525+0000","last_clean":"2023-07-14T12:00:15.487525+0000","last_became_active":"2023-07-14T11:59:42.765536+0000","last_became_peered":"2023-07-14T11:59:42.765536+0000","last_unstale":"2023-07-14T12:00:15.487525+0000","last_undegraded":"2023-07-14T12:00:15.487525+0000","last_fullsized":"2023-07-14T12:00:15.487525+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:27:05.053926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805426+0000","last_change":"2023-07-14T11:59:43.463665+0000","last_active":"2023-07-14T12:00:06.805426+0000","last_peered":"2023-07-14T12:00:06.805426+0000","last_clean":"2023-07-14T12:00:06.805426+0000","last_became_active":"2023-07-14T11:59:43.463368+0000","last_became_peered":"2023-07-14T11:59:43.463368+0000","last_unstale":"2023-07-14T12:00:06.805426+0000","last_undegraded":"2023-07-14T12:00:06.805426+0000","last_fullsized":"2023-07-14T12:00:06.805426+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:49:21.359682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.804608+0000","last_change":"2023-07-14T11:59:47.780706+0000","last_active":"2023-07-14T12:00:06.804608+0000","last_peered":"2023-07-14T12:00:06.804608+0000","last_clean":"2023-07-14T12:00:06.804608+0000","last_became_active":"2023-07-14T11:59:47.780207+0000","last_became_peered":"2023-07-14T11:59:47.780207+0000","last_unstale":"2023-07-14T12:00:06.804608+0000","last_undegraded":"2023-07-14T12:00:06.804608+0000","last_fullsized":"2023-07-14T12:00:06.804608+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:45:25.431762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'14","reported_seq":136,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:01:03.565441+0000","last_change":"2023-07-14T11:59:42.757238+0000","last_active":"2023-07-14T12:01:03.565441+0000","last_peered":"2023-07-14T12:01:03.565441+0000","last_clean":"2023-07-14T12:01:03.565441+0000","last_became_active":"2023-07-14T11:59:42.756887+0000","last_became_peered":"2023-07-14T11:59:42.756887+0000","last_unstale":"2023-07-14T12:01:03.565441+0000","last_undegraded":"2023-07-14T12:01:03.565441+0000","last_fullsized":"2023-07-14T12:01:03.565441+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:09:27.857990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3794,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":96,"num_read_kb":52,"num_write":16,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808529+0000","last_change":"2023-07-14T11:59:43.460741+0000","last_active":"2023-07-14T12:00:06.808529+0000","last_peered":"2023-07-14T12:00:06.808529+0000","last_clean":"2023-07-14T12:00:06.808529+0000","last_became_active":"2023-07-14T11:59:43.460423+0000","last_became_peered":"2023-07-14T11:59:43.460423+0000","last_unstale":"2023-07-14T12:00:06.808529+0000","last_undegraded":"2023-07-14T12:00:06.808529+0000","last_fullsized":"2023-07-14T12:00:06.808529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:26:56.645695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.809254+0000","last_change":"2023-07-14T11:59:47.781321+0000","last_active":"2023-07-14T12:00:06.809254+0000","last_peered":"2023-07-14T12:00:06.809254+0000","last_clean":"2023-07-14T12:00:06.809254+0000","last_became_active":"2023-07-14T11:59:47.780941+0000","last_became_peered":"2023-07-14T11:59:47.780941+0000","last_unstale":"2023-07-14T12:00:06.809254+0000","last_undegraded":"2023-07-14T12:00:06.809254+0000","last_fullsized":"2023-07-14T12:00:06.809254+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:25:26.671430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:17.496520+0000","last_change":"2023-07-14T11:59:42.759403+0000","last_active":"2023-07-14T12:00:17.496520+0000","last_peered":"2023-07-14T12:00:17.496520+0000","last_clean":"2023-07-14T12:00:17.496520+0000","last_became_active":"2023-07-14T11:59:42.759036+0000","last_became_peered":"2023-07-14T11:59:42.759036+0000","last_unstale":"2023-07-14T12:00:17.496520+0000","last_undegraded":"2023-07-14T12:00:17.496520+0000","last_fullsized":"2023-07-14T12:00:17.496520+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:45:13.233768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808124+0000","last_change":"2023-07-14T11:59:43.457644+0000","last_active":"2023-07-14T12:00:06.808124+0000","last_peered":"2023-07-14T12:00:06.808124+0000","last_clean":"2023-07-14T12:00:06.808124+0000","last_became_active":"2023-07-14T11:59:43.457363+0000","last_became_peered":"2023-07-14T11:59:43.457363+0000","last_unstale":"2023-07-14T12:00:06.808124+0000","last_undegraded":"2023-07-14T12:00:06.808124+0000","last_fullsized":"2023-07-14T12:00:06.808124+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:12:17.222392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.812650+0000","last_change":"2023-07-14T11:59:47.789684+0000","last_active":"2023-07-14T12:00:06.812650+0000","last_peered":"2023-07-14T12:00:06.812650+0000","last_clean":"2023-07-14T12:00:06.812650+0000","last_became_active":"2023-07-14T11:59:47.789226+0000","last_became_peered":"2023-07-14T11:59:47.789226+0000","last_unstale":"2023-07-14T12:00:06.812650+0000","last_undegraded":"2023-07-14T12:00:06.812650+0000","last_fullsized":"2023-07-14T12:00:06.812650+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:56:58.961737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:16.494919+0000","last_change":"2023-07-14T11:59:42.751497+0000","last_active":"2023-07-14T12:00:16.494919+0000","last_peered":"2023-07-14T12:00:16.494919+0000","last_clean":"2023-07-14T12:00:16.494919+0000","last_became_active":"2023-07-14T11:59:42.751217+0000","last_became_peered":"2023-07-14T11:59:42.751217+0000","last_unstale":"2023-07-14T12:00:16.494919+0000","last_undegraded":"2023-07-14T12:00:16.494919+0000","last_fullsized":"2023-07-14T12:00:16.494919+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:53:00.140364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.328171+0000","last_change":"2023-07-14T11:59:43.462359+0000","last_active":"2023-07-14T12:00:07.328171+0000","last_peered":"2023-07-14T12:00:07.328171+0000","last_clean":"2023-07-14T12:00:07.328171+0000","last_became_active":"2023-07-14T11:59:43.461781+0000","last_became_peered":"2023-07-14T11:59:43.461781+0000","last_unstale":"2023-07-14T12:00:07.328171+0000","last_undegraded":"2023-07-14T12:00:07.328171+0000","last_fullsized":"2023-07-14T12:00:07.328171+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:10:22.527860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805028+0000","last_change":"2023-07-14T11:59:47.790173+0000","last_active":"2023-07-14T12:00:06.805028+0000","last_peered":"2023-07-14T12:00:06.805028+0000","last_clean":"2023-07-14T12:00:06.805028+0000","last_became_active":"2023-07-14T11:59:47.789418+0000","last_became_peered":"2023-07-14T11:59:47.789418+0000","last_unstale":"2023-07-14T12:00:06.805028+0000","last_undegraded":"2023-07-14T12:00:06.805028+0000","last_fullsized":"2023-07-14T12:00:06.805028+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:54:21.435699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'5","reported_seq":76,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:58.559055+0000","last_change":"2023-07-14T11:59:42.753068+0000","last_active":"2023-07-14T12:00:58.559055+0000","last_peered":"2023-07-14T12:00:58.559055+0000","last_clean":"2023-07-14T12:00:58.559055+0000","last_became_active":"2023-07-14T11:59:42.752794+0000","last_became_peered":"2023-07-14T11:59:42.752794+0000","last_unstale":"2023-07-14T12:00:58.559055+0000","last_undegraded":"2023-07-14T12:00:58.559055+0000","last_fullsized":"2023-07-14T12:00:58.559055+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:31:01.258740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":46,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.328012+0000","last_change":"2023-07-14T11:59:43.468655+0000","last_active":"2023-07-14T12:00:07.328012+0000","last_peered":"2023-07-14T12:00:07.328012+0000","last_clean":"2023-07-14T12:00:07.328012+0000","last_became_active":"2023-07-14T11:59:43.468361+0000","last_became_peered":"2023-07-14T11:59:43.468361+0000","last_unstale":"2023-07-14T12:00:07.328012+0000","last_undegraded":"2023-07-14T12:00:07.328012+0000","last_fullsized":"2023-07-14T12:00:07.328012+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:02:47.181093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808379+0000","last_change":"2023-07-14T11:59:47.783644+0000","last_active":"2023-07-14T12:00:06.808379+0000","last_peered":"2023-07-14T12:00:06.808379+0000","last_clean":"2023-07-14T12:00:06.808379+0000","last_became_active":"2023-07-14T11:59:47.783229+0000","last_became_peered":"2023-07-14T11:59:47.783229+0000","last_unstale":"2023-07-14T12:00:06.808379+0000","last_undegraded":"2023-07-14T12:00:06.808379+0000","last_fullsized":"2023-07-14T12:00:06.808379+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:15:35.788258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:16.494745+0000","last_change":"2023-07-14T11:59:42.768482+0000","last_active":"2023-07-14T12:00:16.494745+0000","last_peered":"2023-07-14T12:00:16.494745+0000","last_clean":"2023-07-14T12:00:16.494745+0000","last_became_active":"2023-07-14T11:59:42.768201+0000","last_became_peered":"2023-07-14T11:59:42.768201+0000","last_unstale":"2023-07-14T12:00:16.494745+0000","last_undegraded":"2023-07-14T12:00:16.494745+0000","last_fullsized":"2023-07-14T12:00:16.494745+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:39:29.438053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808385+0000","last_change":"2023-07-14T11:59:43.469651+0000","last_active":"2023-07-14T12:00:06.808385+0000","last_peered":"2023-07-14T12:00:06.808385+0000","last_clean":"2023-07-14T12:00:06.808385+0000","last_became_active":"2023-07-14T11:59:43.465848+0000","last_became_peered":"2023-07-14T11:59:43.465848+0000","last_unstale":"2023-07-14T12:00:06.808385+0000","last_undegraded":"2023-07-14T12:00:06.808385+0000","last_fullsized":"2023-07-14T12:00:06.808385+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:17:18.313257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.817649+0000","last_change":"2023-07-14T12:00:06.817110+0000","last_active":"2023-07-14T12:00:06.817649+0000","last_peered":"2023-07-14T12:00:06.817649+0000","last_clean":"2023-07-14T12:00:06.817649+0000","last_became_active":"2023-07-14T12:00:06.816587+0000","last_became_peered":"2023-07-14T12:00:06.816587+0000","last_unstale":"2023-07-14T12:00:06.817649+0000","last_undegraded":"2023-07-14T12:00:06.817649+0000","last_fullsized":"2023-07-14T12:00:06.817649+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:04:52.873643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,0,3,1],"acting":[10,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:17.497708+0000","last_change":"2023-07-14T11:59:42.758106+0000","last_active":"2023-07-14T12:00:17.497708+0000","last_peered":"2023-07-14T12:00:17.497708+0000","last_clean":"2023-07-14T12:00:17.497708+0000","last_became_active":"2023-07-14T11:59:42.757840+0000","last_became_peered":"2023-07-14T11:59:42.757840+0000","last_unstale":"2023-07-14T12:00:17.497708+0000","last_undegraded":"2023-07-14T12:00:17.497708+0000","last_fullsized":"2023-07-14T12:00:17.497708+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:12:46.164720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.812652+0000","last_change":"2023-07-14T11:59:43.467800+0000","last_active":"2023-07-14T12:00:06.812652+0000","last_peered":"2023-07-14T12:00:06.812652+0000","last_clean":"2023-07-14T12:00:06.812652+0000","last_became_active":"2023-07-14T11:59:43.467448+0000","last_became_peered":"2023-07-14T11:59:43.467448+0000","last_unstale":"2023-07-14T12:00:06.812652+0000","last_undegraded":"2023-07-14T12:00:06.812652+0000","last_fullsized":"2023-07-14T12:00:06.812652+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:34:25.781268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.807574+0000","last_change":"2023-07-14T11:59:47.915884+0000","last_active":"2023-07-14T12:00:06.807574+0000","last_peered":"2023-07-14T12:00:06.807574+0000","last_clean":"2023-07-14T12:00:06.807574+0000","last_became_active":"2023-07-14T11:59:47.915449+0000","last_became_peered":"2023-07-14T11:59:47.915449+0000","last_unstale":"2023-07-14T12:00:06.807574+0000","last_undegraded":"2023-07-14T12:00:06.807574+0000","last_fullsized":"2023-07-14T12:00:06.807574+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:41:55.813157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.804996+0000","last_change":"2023-07-14T11:59:42.771581+0000","last_active":"2023-07-14T12:00:06.804996+0000","last_peered":"2023-07-14T12:00:06.804996+0000","last_clean":"2023-07-14T12:00:06.804996+0000","last_became_active":"2023-07-14T11:59:42.771228+0000","last_became_peered":"2023-07-14T11:59:42.771228+0000","last_unstale":"2023-07-14T12:00:06.804996+0000","last_undegraded":"2023-07-14T12:00:06.804996+0000","last_fullsized":"2023-07-14T12:00:06.804996+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:32:35.711544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808751+0000","last_change":"2023-07-14T11:59:43.460987+0000","last_active":"2023-07-14T12:00:06.808751+0000","last_peered":"2023-07-14T12:00:06.808751+0000","last_clean":"2023-07-14T12:00:06.808751+0000","last_became_active":"2023-07-14T11:59:43.460604+0000","last_became_peered":"2023-07-14T11:59:43.460604+0000","last_unstale":"2023-07-14T12:00:06.808751+0000","last_undegraded":"2023-07-14T12:00:06.808751+0000","last_fullsized":"2023-07-14T12:00:06.808751+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:00:15.767598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808575+0000","last_change":"2023-07-14T11:59:47.788604+0000","last_active":"2023-07-14T12:00:06.808575+0000","last_peered":"2023-07-14T12:00:06.808575+0000","last_clean":"2023-07-14T12:00:06.808575+0000","last_became_active":"2023-07-14T11:59:47.787854+0000","last_became_peered":"2023-07-14T11:59:47.787854+0000","last_unstale":"2023-07-14T12:00:06.808575+0000","last_undegraded":"2023-07-14T12:00:06.808575+0000","last_fullsized":"2023-07-14T12:00:06.808575+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:19:18.510764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'5","reported_seq":79,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:01:02.564666+0000","last_change":"2023-07-14T11:59:42.768985+0000","last_active":"2023-07-14T12:01:02.564666+0000","last_peered":"2023-07-14T12:01:02.564666+0000","last_clean":"2023-07-14T12:01:02.564666+0000","last_became_active":"2023-07-14T11:59:42.768688+0000","last_became_peered":"2023-07-14T11:59:42.768688+0000","last_unstale":"2023-07-14T12:01:02.564666+0000","last_undegraded":"2023-07-14T12:01:02.564666+0000","last_fullsized":"2023-07-14T12:01:02.564666+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:20:55.661084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":48,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.807718+0000","last_change":"2023-07-14T11:59:43.465529+0000","last_active":"2023-07-14T12:00:06.807718+0000","last_peered":"2023-07-14T12:00:06.807718+0000","last_clean":"2023-07-14T12:00:06.807718+0000","last_became_active":"2023-07-14T11:59:43.465206+0000","last_became_peered":"2023-07-14T11:59:43.465206+0000","last_unstale":"2023-07-14T12:00:06.807718+0000","last_undegraded":"2023-07-14T12:00:06.807718+0000","last_fullsized":"2023-07-14T12:00:06.807718+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:52:33.760607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808309+0000","last_change":"2023-07-14T11:59:47.915842+0000","last_active":"2023-07-14T12:00:06.808309+0000","last_peered":"2023-07-14T12:00:06.808309+0000","last_clean":"2023-07-14T12:00:06.808309+0000","last_became_active":"2023-07-14T11:59:47.915304+0000","last_became_peered":"2023-07-14T11:59:47.915304+0000","last_unstale":"2023-07-14T12:00:06.808309+0000","last_undegraded":"2023-07-14T12:00:06.808309+0000","last_fullsized":"2023-07-14T12:00:06.808309+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:26:43.397241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.809764+0000","last_change":"2023-07-14T11:59:43.472296+0000","last_active":"2023-07-14T12:00:06.809764+0000","last_peered":"2023-07-14T12:00:06.809764+0000","last_clean":"2023-07-14T12:00:06.809764+0000","last_became_active":"2023-07-14T11:59:43.470569+0000","last_became_peered":"2023-07-14T11:59:43.470569+0000","last_unstale":"2023-07-14T12:00:06.809764+0000","last_undegraded":"2023-07-14T12:00:06.809764+0000","last_fullsized":"2023-07-14T12:00:06.809764+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:06:23.546910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":77,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:01:00.561035+0000","last_change":"2023-07-14T11:59:42.769225+0000","last_active":"2023-07-14T12:01:00.561035+0000","last_peered":"2023-07-14T12:01:00.561035+0000","last_clean":"2023-07-14T12:01:00.561035+0000","last_became_active":"2023-07-14T11:59:42.768877+0000","last_became_peered":"2023-07-14T11:59:42.768877+0000","last_unstale":"2023-07-14T12:01:00.561035+0000","last_undegraded":"2023-07-14T12:01:00.561035+0000","last_fullsized":"2023-07-14T12:01:00.561035+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:19:23.782648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":46,"num_read_kb":46,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.328375+0000","last_change":"2023-07-14T11:59:47.792665+0000","last_active":"2023-07-14T12:00:07.328375+0000","last_peered":"2023-07-14T12:00:07.328375+0000","last_clean":"2023-07-14T12:00:07.328375+0000","last_became_active":"2023-07-14T11:59:47.792308+0000","last_became_peered":"2023-07-14T11:59:47.792308+0000","last_unstale":"2023-07-14T12:00:07.328375+0000","last_undegraded":"2023-07-14T12:00:07.328375+0000","last_fullsized":"2023-07-14T12:00:07.328375+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:18:05.628049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.810583+0000","last_change":"2023-07-14T11:59:43.461103+0000","last_active":"2023-07-14T12:00:06.810583+0000","last_peered":"2023-07-14T12:00:06.810583+0000","last_clean":"2023-07-14T12:00:06.810583+0000","last_became_active":"2023-07-14T11:59:43.460756+0000","last_became_peered":"2023-07-14T11:59:43.460756+0000","last_unstale":"2023-07-14T12:00:06.810583+0000","last_undegraded":"2023-07-14T12:00:06.810583+0000","last_fullsized":"2023-07-14T12:00:06.810583+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:05:41.752001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:17.497582+0000","last_change":"2023-07-14T11:59:42.752680+0000","last_active":"2023-07-14T12:00:17.497582+0000","last_peered":"2023-07-14T12:00:17.497582+0000","last_clean":"2023-07-14T12:00:17.497582+0000","last_became_active":"2023-07-14T11:59:42.752305+0000","last_became_peered":"2023-07-14T11:59:42.752305+0000","last_unstale":"2023-07-14T12:00:17.497582+0000","last_undegraded":"2023-07-14T12:00:17.497582+0000","last_fullsized":"2023-07-14T12:00:17.497582+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:01:57.359059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":353,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.327820+0000","last_change":"2023-07-14T11:58:37.124871+0000","last_active":"2023-07-14T12:00:07.327820+0000","last_peered":"2023-07-14T12:00:07.327820+0000","last_clean":"2023-07-14T12:00:07.327820+0000","last_became_active":"2023-07-14T11:58:37.124398+0000","last_became_peered":"2023-07-14T11:58:37.124398+0000","last_unstale":"2023-07-14T12:00:07.327820+0000","last_undegraded":"2023-07-14T12:00:07.327820+0000","last_fullsized":"2023-07-14T12:00:07.327820+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:55:28.287160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:55:28.287160+0000","last_clean_scrub_stamp":"2023-07-14T11:55:28.287160+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:00:38.717405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.815687+0000","last_change":"2023-07-14T12:00:06.815687+0000","last_active":"2023-07-14T12:00:06.815687+0000","last_peered":"2023-07-14T12:00:06.815687+0000","last_clean":"2023-07-14T12:00:06.815687+0000","last_became_active":"2023-07-14T12:00:06.815281+0000","last_became_peered":"2023-07-14T12:00:06.815281+0000","last_unstale":"2023-07-14T12:00:06.815687+0000","last_undegraded":"2023-07-14T12:00:06.815687+0000","last_fullsized":"2023-07-14T12:00:06.815687+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:04:42.469562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.813952+0000","last_change":"2023-07-14T12:00:06.813952+0000","last_active":"2023-07-14T12:00:06.813952+0000","last_peered":"2023-07-14T12:00:06.813952+0000","last_clean":"2023-07-14T12:00:06.813952+0000","last_became_active":"2023-07-14T12:00:06.813581+0000","last_became_peered":"2023-07-14T12:00:06.813581+0000","last_unstale":"2023-07-14T12:00:06.813952+0000","last_undegraded":"2023-07-14T12:00:06.813952+0000","last_fullsized":"2023-07-14T12:00:06.813952+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:12:06.783097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:14.479090+0000","last_change":"2023-07-14T11:59:42.757400+0000","last_active":"2023-07-14T12:00:14.479090+0000","last_peered":"2023-07-14T12:00:14.479090+0000","last_clean":"2023-07-14T12:00:14.479090+0000","last_became_active":"2023-07-14T11:59:42.757047+0000","last_became_peered":"2023-07-14T11:59:42.757047+0000","last_unstale":"2023-07-14T12:00:14.479090+0000","last_undegraded":"2023-07-14T12:00:14.479090+0000","last_fullsized":"2023-07-14T12:00:14.479090+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:42:39.292073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.811396+0000","last_change":"2023-07-14T11:59:47.788486+0000","last_active":"2023-07-14T12:00:06.811396+0000","last_peered":"2023-07-14T12:00:06.811396+0000","last_clean":"2023-07-14T12:00:06.811396+0000","last_became_active":"2023-07-14T11:59:47.787995+0000","last_became_peered":"2023-07-14T11:59:47.787995+0000","last_unstale":"2023-07-14T12:00:06.811396+0000","last_undegraded":"2023-07-14T12:00:06.811396+0000","last_fullsized":"2023-07-14T12:00:06.811396+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:01:52.099573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:16.495497+0000","last_change":"2023-07-14T11:59:42.765810+0000","last_active":"2023-07-14T12:00:16.495497+0000","last_peered":"2023-07-14T12:00:16.495497+0000","last_clean":"2023-07-14T12:00:16.495497+0000","last_became_active":"2023-07-14T11:59:42.765528+0000","last_became_peered":"2023-07-14T11:59:42.765528+0000","last_unstale":"2023-07-14T12:00:16.495497+0000","last_undegraded":"2023-07-14T12:00:16.495497+0000","last_fullsized":"2023-07-14T12:00:16.495497+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:18:02.321077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808724+0000","last_change":"2023-07-14T11:59:43.463763+0000","last_active":"2023-07-14T12:00:06.808724+0000","last_peered":"2023-07-14T12:00:06.808724+0000","last_clean":"2023-07-14T12:00:06.808724+0000","last_became_active":"2023-07-14T11:59:43.463411+0000","last_became_peered":"2023-07-14T11:59:43.463411+0000","last_unstale":"2023-07-14T12:00:06.808724+0000","last_undegraded":"2023-07-14T12:00:06.808724+0000","last_fullsized":"2023-07-14T12:00:06.808724+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:57:24.209856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.809720+0000","last_change":"2023-07-14T11:59:47.916405+0000","last_active":"2023-07-14T12:00:06.809720+0000","last_peered":"2023-07-14T12:00:06.809720+0000","last_clean":"2023-07-14T12:00:06.809720+0000","last_became_active":"2023-07-14T11:59:47.915892+0000","last_became_peered":"2023-07-14T11:59:47.915892+0000","last_unstale":"2023-07-14T12:00:06.809720+0000","last_undegraded":"2023-07-14T12:00:06.809720+0000","last_fullsized":"2023-07-14T12:00:06.809720+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:58:28.977188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'7","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:15.491661+0000","last_change":"2023-07-14T11:59:42.754848+0000","last_active":"2023-07-14T12:00:15.491661+0000","last_peered":"2023-07-14T12:00:15.491661+0000","last_clean":"2023-07-14T12:00:15.491661+0000","last_became_active":"2023-07-14T11:59:42.754522+0000","last_became_peered":"2023-07-14T11:59:42.754522+0000","last_unstale":"2023-07-14T12:00:15.491661+0000","last_undegraded":"2023-07-14T12:00:15.491661+0000","last_fullsized":"2023-07-14T12:00:15.491661+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:49:48.659717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.815209+0000","last_change":"2023-07-14T11:59:43.458954+0000","last_active":"2023-07-14T12:00:06.815209+0000","last_peered":"2023-07-14T12:00:06.815209+0000","last_clean":"2023-07-14T12:00:06.815209+0000","last_became_active":"2023-07-14T11:59:43.458631+0000","last_became_peered":"2023-07-14T11:59:43.458631+0000","last_unstale":"2023-07-14T12:00:06.815209+0000","last_undegraded":"2023-07-14T12:00:06.815209+0000","last_fullsized":"2023-07-14T12:00:06.815209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:41:19.853895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805050+0000","last_change":"2023-07-14T11:59:47.776879+0000","last_active":"2023-07-14T12:00:06.805050+0000","last_peered":"2023-07-14T12:00:06.805050+0000","last_clean":"2023-07-14T12:00:06.805050+0000","last_became_active":"2023-07-14T11:59:47.776514+0000","last_became_peered":"2023-07-14T11:59:47.776514+0000","last_unstale":"2023-07-14T12:00:06.805050+0000","last_undegraded":"2023-07-14T12:00:06.805050+0000","last_fullsized":"2023-07-14T12:00:06.805050+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:14:32.460215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:16.496298+0000","last_change":"2023-07-14T11:59:42.754657+0000","last_active":"2023-07-14T12:00:16.496298+0000","last_peered":"2023-07-14T12:00:16.496298+0000","last_clean":"2023-07-14T12:00:16.496298+0000","last_became_active":"2023-07-14T11:59:42.754355+0000","last_became_peered":"2023-07-14T11:59:42.754355+0000","last_unstale":"2023-07-14T12:00:16.496298+0000","last_undegraded":"2023-07-14T12:00:16.496298+0000","last_fullsized":"2023-07-14T12:00:16.496298+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:16:59.044903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.329782+0000","last_change":"2023-07-14T12:00:07.329782+0000","last_active":"2023-07-14T12:00:07.329782+0000","last_peered":"2023-07-14T12:00:07.329782+0000","last_clean":"2023-07-14T12:00:07.329782+0000","last_became_active":"2023-07-14T12:00:07.329274+0000","last_became_peered":"2023-07-14T12:00:07.329274+0000","last_unstale":"2023-07-14T12:00:07.329782+0000","last_undegraded":"2023-07-14T12:00:07.329782+0000","last_fullsized":"2023-07-14T12:00:07.329782+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:49:01.837874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.807313+0000","last_change":"2023-07-14T11:59:47.916411+0000","last_active":"2023-07-14T12:00:06.807313+0000","last_peered":"2023-07-14T12:00:06.807313+0000","last_clean":"2023-07-14T12:00:06.807313+0000","last_became_active":"2023-07-14T11:59:47.915877+0000","last_became_peered":"2023-07-14T11:59:47.915877+0000","last_unstale":"2023-07-14T12:00:06.807313+0000","last_undegraded":"2023-07-14T12:00:06.807313+0000","last_fullsized":"2023-07-14T12:00:06.807313+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:08:05.471264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:13.463576+0000","last_change":"2023-07-14T11:59:42.769080+0000","last_active":"2023-07-14T12:00:13.463576+0000","last_peered":"2023-07-14T12:00:13.463576+0000","last_clean":"2023-07-14T12:00:13.463576+0000","last_became_active":"2023-07-14T11:59:42.768782+0000","last_became_peered":"2023-07-14T11:59:42.768782+0000","last_unstale":"2023-07-14T12:00:13.463576+0000","last_undegraded":"2023-07-14T12:00:13.463576+0000","last_fullsized":"2023-07-14T12:00:13.463576+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:31:48.128002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.806874+0000","last_change":"2023-07-14T11:59:43.460284+0000","last_active":"2023-07-14T12:00:06.806874+0000","last_peered":"2023-07-14T12:00:06.806874+0000","last_clean":"2023-07-14T12:00:06.806874+0000","last_became_active":"2023-07-14T11:59:43.459979+0000","last_became_peered":"2023-07-14T11:59:43.459979+0000","last_unstale":"2023-07-14T12:00:06.806874+0000","last_undegraded":"2023-07-14T12:00:06.806874+0000","last_fullsized":"2023-07-14T12:00:06.806874+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:25:15.513703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805201+0000","last_change":"2023-07-14T11:59:47.782359+0000","last_active":"2023-07-14T12:00:06.805201+0000","last_peered":"2023-07-14T12:00:06.805201+0000","last_clean":"2023-07-14T12:00:06.805201+0000","last_became_active":"2023-07-14T11:59:47.780484+0000","last_became_peered":"2023-07-14T11:59:47.780484+0000","last_unstale":"2023-07-14T12:00:06.805201+0000","last_undegraded":"2023-07-14T12:00:06.805201+0000","last_fullsized":"2023-07-14T12:00:06.805201+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:24:16.054488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:16.495085+0000","last_change":"2023-07-14T11:59:42.756823+0000","last_active":"2023-07-14T12:00:16.495085+0000","last_peered":"2023-07-14T12:00:16.495085+0000","last_clean":"2023-07-14T12:00:16.495085+0000","last_became_active":"2023-07-14T11:59:42.756412+0000","last_became_peered":"2023-07-14T11:59:42.756412+0000","last_unstale":"2023-07-14T12:00:16.495085+0000","last_undegraded":"2023-07-14T12:00:16.495085+0000","last_fullsized":"2023-07-14T12:00:16.495085+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:45:25.674809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.811156+0000","last_change":"2023-07-14T11:59:43.469151+0000","last_active":"2023-07-14T12:00:06.811156+0000","last_peered":"2023-07-14T12:00:06.811156+0000","last_clean":"2023-07-14T12:00:06.811156+0000","last_became_active":"2023-07-14T11:59:43.468835+0000","last_became_peered":"2023-07-14T11:59:43.468835+0000","last_unstale":"2023-07-14T12:00:06.811156+0000","last_undegraded":"2023-07-14T12:00:06.811156+0000","last_fullsized":"2023-07-14T12:00:06.811156+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:01:00.517226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.807655+0000","last_change":"2023-07-14T11:59:47.782663+0000","last_active":"2023-07-14T12:00:06.807655+0000","last_peered":"2023-07-14T12:00:06.807655+0000","last_clean":"2023-07-14T12:00:06.807655+0000","last_became_active":"2023-07-14T11:59:47.782175+0000","last_became_peered":"2023-07-14T11:59:47.782175+0000","last_unstale":"2023-07-14T12:00:06.807655+0000","last_undegraded":"2023-07-14T12:00:06.807655+0000","last_fullsized":"2023-07-14T12:00:06.807655+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:43:48.229057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:16.994892+0000","last_change":"2023-07-14T11:59:42.759809+0000","last_active":"2023-07-14T12:00:16.994892+0000","last_peered":"2023-07-14T12:00:16.994892+0000","last_clean":"2023-07-14T12:00:16.994892+0000","last_became_active":"2023-07-14T11:59:42.759493+0000","last_became_peered":"2023-07-14T11:59:42.759493+0000","last_unstale":"2023-07-14T12:00:16.994892+0000","last_undegraded":"2023-07-14T12:00:16.994892+0000","last_fullsized":"2023-07-14T12:00:16.994892+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:21:22.391310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.810840+0000","last_change":"2023-07-14T11:59:43.465308+0000","last_active":"2023-07-14T12:00:06.810840+0000","last_peered":"2023-07-14T12:00:06.810840+0000","last_clean":"2023-07-14T12:00:06.810840+0000","last_became_active":"2023-07-14T11:59:43.464995+0000","last_became_peered":"2023-07-14T11:59:43.464995+0000","last_unstale":"2023-07-14T12:00:06.810840+0000","last_undegraded":"2023-07-14T12:00:06.810840+0000","last_fullsized":"2023-07-14T12:00:06.810840+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:22:41.954399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.807244+0000","last_change":"2023-07-14T11:59:47.916435+0000","last_active":"2023-07-14T12:00:06.807244+0000","last_peered":"2023-07-14T12:00:06.807244+0000","last_clean":"2023-07-14T12:00:06.807244+0000","last_became_active":"2023-07-14T11:59:47.915894+0000","last_became_peered":"2023-07-14T11:59:47.915894+0000","last_unstale":"2023-07-14T12:00:06.807244+0000","last_undegraded":"2023-07-14T12:00:06.807244+0000","last_fullsized":"2023-07-14T12:00:06.807244+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:15:10.755288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:13.466102+0000","last_change":"2023-07-14T11:59:42.760422+0000","last_active":"2023-07-14T12:00:13.466102+0000","last_peered":"2023-07-14T12:00:13.466102+0000","last_clean":"2023-07-14T12:00:13.466102+0000","last_became_active":"2023-07-14T11:59:42.758548+0000","last_became_peered":"2023-07-14T11:59:42.758548+0000","last_unstale":"2023-07-14T12:00:13.466102+0000","last_undegraded":"2023-07-14T12:00:13.466102+0000","last_fullsized":"2023-07-14T12:00:13.466102+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:00:42.799391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.328110+0000","last_change":"2023-07-14T11:59:43.455296+0000","last_active":"2023-07-14T12:00:07.328110+0000","last_peered":"2023-07-14T12:00:07.328110+0000","last_clean":"2023-07-14T12:00:07.328110+0000","last_became_active":"2023-07-14T11:59:43.455003+0000","last_became_peered":"2023-07-14T11:59:43.455003+0000","last_unstale":"2023-07-14T12:00:07.328110+0000","last_undegraded":"2023-07-14T12:00:07.328110+0000","last_fullsized":"2023-07-14T12:00:07.328110+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:43:29.555694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805023+0000","last_change":"2023-07-14T11:59:47.779262+0000","last_active":"2023-07-14T12:00:06.805023+0000","last_peered":"2023-07-14T12:00:06.805023+0000","last_clean":"2023-07-14T12:00:06.805023+0000","last_became_active":"2023-07-14T11:59:47.778863+0000","last_became_peered":"2023-07-14T11:59:47.778863+0000","last_unstale":"2023-07-14T12:00:06.805023+0000","last_undegraded":"2023-07-14T12:00:06.805023+0000","last_fullsized":"2023-07-14T12:00:06.805023+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:11:16.752296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'5","reported_seq":78,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:01:00.561951+0000","last_change":"2023-07-14T11:59:42.768252+0000","last_active":"2023-07-14T12:01:00.561951+0000","last_peered":"2023-07-14T12:01:00.561951+0000","last_clean":"2023-07-14T12:01:00.561951+0000","last_became_active":"2023-07-14T11:59:42.767956+0000","last_became_peered":"2023-07-14T11:59:42.767956+0000","last_unstale":"2023-07-14T12:01:00.561951+0000","last_undegraded":"2023-07-14T12:01:00.561951+0000","last_fullsized":"2023-07-14T12:01:00.561951+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:19:59.247002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":47,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.328734+0000","last_change":"2023-07-14T11:59:43.461899+0000","last_active":"2023-07-14T12:00:07.328734+0000","last_peered":"2023-07-14T12:00:07.328734+0000","last_clean":"2023-07-14T12:00:07.328734+0000","last_became_active":"2023-07-14T11:59:43.461535+0000","last_became_peered":"2023-07-14T11:59:43.461535+0000","last_unstale":"2023-07-14T12:00:07.328734+0000","last_undegraded":"2023-07-14T12:00:07.328734+0000","last_fullsized":"2023-07-14T12:00:07.328734+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:43:24.603371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.812720+0000","last_change":"2023-07-14T11:59:47.782964+0000","last_active":"2023-07-14T12:00:06.812720+0000","last_peered":"2023-07-14T12:00:06.812720+0000","last_clean":"2023-07-14T12:00:06.812720+0000","last_became_active":"2023-07-14T11:59:47.782563+0000","last_became_peered":"2023-07-14T11:59:47.782563+0000","last_unstale":"2023-07-14T12:00:06.812720+0000","last_undegraded":"2023-07-14T12:00:06.812720+0000","last_fullsized":"2023-07-14T12:00:06.812720+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:27:49.195128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:16.494770+0000","last_change":"2023-07-14T11:59:42.751355+0000","last_active":"2023-07-14T12:00:16.494770+0000","last_peered":"2023-07-14T12:00:16.494770+0000","last_clean":"2023-07-14T12:00:16.494770+0000","last_became_active":"2023-07-14T11:59:42.751063+0000","last_became_peered":"2023-07-14T11:59:42.751063+0000","last_unstale":"2023-07-14T12:00:16.494770+0000","last_undegraded":"2023-07-14T12:00:16.494770+0000","last_fullsized":"2023-07-14T12:00:16.494770+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:28:15.966689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.328252+0000","last_change":"2023-07-14T11:59:43.465891+0000","last_active":"2023-07-14T12:00:07.328252+0000","last_peered":"2023-07-14T12:00:07.328252+0000","last_clean":"2023-07-14T12:00:07.328252+0000","last_became_active":"2023-07-14T11:59:43.465575+0000","last_became_peered":"2023-07-14T11:59:43.465575+0000","last_unstale":"2023-07-14T12:00:07.328252+0000","last_undegraded":"2023-07-14T12:00:07.328252+0000","last_fullsized":"2023-07-14T12:00:07.328252+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:43:52.704477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.809069+0000","last_change":"2023-07-14T11:59:47.777547+0000","last_active":"2023-07-14T12:00:06.809069+0000","last_peered":"2023-07-14T12:00:06.809069+0000","last_clean":"2023-07-14T12:00:06.809069+0000","last_became_active":"2023-07-14T11:59:47.777150+0000","last_became_peered":"2023-07-14T11:59:47.777150+0000","last_unstale":"2023-07-14T12:00:06.809069+0000","last_undegraded":"2023-07-14T12:00:06.809069+0000","last_fullsized":"2023-07-14T12:00:06.809069+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:21:35.638026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:16.494997+0000","last_change":"2023-07-14T11:59:42.761236+0000","last_active":"2023-07-14T12:00:16.494997+0000","last_peered":"2023-07-14T12:00:16.494997+0000","last_clean":"2023-07-14T12:00:16.494997+0000","last_became_active":"2023-07-14T11:59:42.760893+0000","last_became_peered":"2023-07-14T11:59:42.760893+0000","last_unstale":"2023-07-14T12:00:16.494997+0000","last_undegraded":"2023-07-14T12:00:16.494997+0000","last_fullsized":"2023-07-14T12:00:16.494997+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:11:17.477118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.328670+0000","last_change":"2023-07-14T11:59:43.462518+0000","last_active":"2023-07-14T12:00:07.328670+0000","last_peered":"2023-07-14T12:00:07.328670+0000","last_clean":"2023-07-14T12:00:07.328670+0000","last_became_active":"2023-07-14T11:59:43.462178+0000","last_became_peered":"2023-07-14T11:59:43.462178+0000","last_unstale":"2023-07-14T12:00:07.328670+0000","last_undegraded":"2023-07-14T12:00:07.328670+0000","last_fullsized":"2023-07-14T12:00:07.328670+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:09:38.398771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.811302+0000","last_change":"2023-07-14T11:59:43.474142+0000","last_active":"2023-07-14T12:00:06.811302+0000","last_peered":"2023-07-14T12:00:06.811302+0000","last_clean":"2023-07-14T12:00:06.811302+0000","last_became_active":"2023-07-14T11:59:43.473863+0000","last_became_peered":"2023-07-14T11:59:43.473863+0000","last_unstale":"2023-07-14T12:00:06.811302+0000","last_undegraded":"2023-07-14T12:00:06.811302+0000","last_fullsized":"2023-07-14T12:00:06.811302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:34:40.849796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805367+0000","last_change":"2023-07-14T11:59:43.462193+0000","last_active":"2023-07-14T12:00:06.805367+0000","last_peered":"2023-07-14T12:00:06.805367+0000","last_clean":"2023-07-14T12:00:06.805367+0000","last_became_active":"2023-07-14T11:59:43.461390+0000","last_became_peered":"2023-07-14T11:59:43.461390+0000","last_unstale":"2023-07-14T12:00:06.805367+0000","last_undegraded":"2023-07-14T12:00:06.805367+0000","last_fullsized":"2023-07-14T12:00:06.805367+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:43:44.553075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805098+0000","last_change":"2023-07-14T11:59:43.467561+0000","last_active":"2023-07-14T12:00:06.805098+0000","last_peered":"2023-07-14T12:00:06.805098+0000","last_clean":"2023-07-14T12:00:06.805098+0000","last_became_active":"2023-07-14T11:59:43.467256+0000","last_became_peered":"2023-07-14T11:59:43.467256+0000","last_unstale":"2023-07-14T12:00:06.805098+0000","last_undegraded":"2023-07-14T12:00:06.805098+0000","last_fullsized":"2023-07-14T12:00:06.805098+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:04:06.326191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.813046+0000","last_change":"2023-07-14T11:59:43.472405+0000","last_active":"2023-07-14T12:00:06.813046+0000","last_peered":"2023-07-14T12:00:06.813046+0000","last_clean":"2023-07-14T12:00:06.813046+0000","last_became_active":"2023-07-14T11:59:43.471863+0000","last_became_peered":"2023-07-14T11:59:43.471863+0000","last_unstale":"2023-07-14T12:00:06.813046+0000","last_undegraded":"2023-07-14T12:00:06.813046+0000","last_fullsized":"2023-07-14T12:00:06.813046+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:22:54.987390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808865+0000","last_change":"2023-07-14T11:59:43.469966+0000","last_active":"2023-07-14T12:00:06.808865+0000","last_peered":"2023-07-14T12:00:06.808865+0000","last_clean":"2023-07-14T12:00:06.808865+0000","last_became_active":"2023-07-14T11:59:43.469540+0000","last_became_peered":"2023-07-14T11:59:43.469540+0000","last_unstale":"2023-07-14T12:00:06.808865+0000","last_undegraded":"2023-07-14T12:00:06.808865+0000","last_fullsized":"2023-07-14T12:00:06.808865+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:19:04.529881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.812535+0000","last_change":"2023-07-14T11:59:43.465032+0000","last_active":"2023-07-14T12:00:06.812535+0000","last_peered":"2023-07-14T12:00:06.812535+0000","last_clean":"2023-07-14T12:00:06.812535+0000","last_became_active":"2023-07-14T11:59:43.464689+0000","last_became_peered":"2023-07-14T11:59:43.464689+0000","last_unstale":"2023-07-14T12:00:06.812535+0000","last_undegraded":"2023-07-14T12:00:06.812535+0000","last_fullsized":"2023-07-14T12:00:06.812535+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:15:12.845214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.806463+0000","last_change":"2023-07-14T11:59:43.463042+0000","last_active":"2023-07-14T12:00:06.806463+0000","last_peered":"2023-07-14T12:00:06.806463+0000","last_clean":"2023-07-14T12:00:06.806463+0000","last_became_active":"2023-07-14T11:59:43.462665+0000","last_became_peered":"2023-07-14T11:59:43.462665+0000","last_unstale":"2023-07-14T12:00:06.806463+0000","last_undegraded":"2023-07-14T12:00:06.806463+0000","last_fullsized":"2023-07-14T12:00:06.806463+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:50:19.141988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.816236+0000","last_change":"2023-07-14T12:00:06.816236+0000","last_active":"2023-07-14T12:00:06.816236+0000","last_peered":"2023-07-14T12:00:06.816236+0000","last_clean":"2023-07-14T12:00:06.816236+0000","last_became_active":"2023-07-14T12:00:06.815733+0000","last_became_peered":"2023-07-14T12:00:06.815733+0000","last_unstale":"2023-07-14T12:00:06.816236+0000","last_undegraded":"2023-07-14T12:00:06.816236+0000","last_fullsized":"2023-07-14T12:00:06.816236+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:14:57.114402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.809719+0000","last_change":"2023-07-14T11:59:43.470256+0000","last_active":"2023-07-14T12:00:06.809719+0000","last_peered":"2023-07-14T12:00:06.809719+0000","last_clean":"2023-07-14T12:00:06.809719+0000","last_became_active":"2023-07-14T11:59:43.469861+0000","last_became_peered":"2023-07-14T11:59:43.469861+0000","last_unstale":"2023-07-14T12:00:06.809719+0000","last_undegraded":"2023-07-14T12:00:06.809719+0000","last_fullsized":"2023-07-14T12:00:06.809719+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:54:04.221673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.811456+0000","last_change":"2023-07-14T11:59:43.458009+0000","last_active":"2023-07-14T12:00:06.811456+0000","last_peered":"2023-07-14T12:00:06.811456+0000","last_clean":"2023-07-14T12:00:06.811456+0000","last_became_active":"2023-07-14T11:59:43.457657+0000","last_became_peered":"2023-07-14T11:59:43.457657+0000","last_unstale":"2023-07-14T12:00:06.811456+0000","last_undegraded":"2023-07-14T12:00:06.811456+0000","last_fullsized":"2023-07-14T12:00:06.811456+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:14:41.854547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805358+0000","last_change":"2023-07-14T11:59:43.457568+0000","last_active":"2023-07-14T12:00:06.805358+0000","last_peered":"2023-07-14T12:00:06.805358+0000","last_clean":"2023-07-14T12:00:06.805358+0000","last_became_active":"2023-07-14T11:59:43.457203+0000","last_became_peered":"2023-07-14T11:59:43.457203+0000","last_unstale":"2023-07-14T12:00:06.805358+0000","last_undegraded":"2023-07-14T12:00:06.805358+0000","last_fullsized":"2023-07-14T12:00:06.805358+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:14:37.939509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808045+0000","last_change":"2023-07-14T11:59:43.470840+0000","last_active":"2023-07-14T12:00:06.808045+0000","last_peered":"2023-07-14T12:00:06.808045+0000","last_clean":"2023-07-14T12:00:06.808045+0000","last_became_active":"2023-07-14T11:59:43.470531+0000","last_became_peered":"2023-07-14T11:59:43.470531+0000","last_unstale":"2023-07-14T12:00:06.808045+0000","last_undegraded":"2023-07-14T12:00:06.808045+0000","last_fullsized":"2023-07-14T12:00:06.808045+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:49:50.862808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.817057+0000","last_change":"2023-07-14T12:00:06.817057+0000","last_active":"2023-07-14T12:00:06.817057+0000","last_peered":"2023-07-14T12:00:06.817057+0000","last_clean":"2023-07-14T12:00:06.817057+0000","last_became_active":"2023-07-14T12:00:06.816680+0000","last_became_peered":"2023-07-14T12:00:06.816680+0000","last_unstale":"2023-07-14T12:00:06.817057+0000","last_undegraded":"2023-07-14T12:00:06.817057+0000","last_fullsized":"2023-07-14T12:00:06.817057+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:22:01.363770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284390,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":10024,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":488,"num_read_kb":266,"num_write":209,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":30072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416412,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":983040,"data_stored":487359,"data_compressed":17563,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 12:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.58499999999999996,"15min":0.59099999999999997},"min":{"1min":0.34599999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.503},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.54500000000000004,"15min":0.55100000000000005},"min":{"1min":0.38300000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.72499999999999998,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.54500000000000004}]},{"osd":1,"last update":"Fri Jul 14 12:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82499999999999996,"5min":1.048,"15min":1.048},"last":0.753},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.60999999999999999,"15min":0.60199999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.70399999999999996}]},{"osd":2,"last update":"Fri Jul 14 12:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.499,"15min":0.498},"min":{"1min":0.317,"5min":0.253,"15min":0.253},"max":{"1min":0.755,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.502,"15min":0.496},"min":{"1min":0.34100000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.78500000000000003,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.62}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":4,"last update":"Fri Jul 14 12:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.68600000000000005,"15min":0.67600000000000005},"min":{"1min":0.5,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.879,"5min":1.149,"15min":1.149},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.64400000000000002,"15min":0.64100000000000001},"min":{"1min":0.42199999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81299999999999994,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.76600000000000001}]},{"osd":5,"last update":"Fri Jul 14 12:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56100000000000005,"15min":0.56299999999999994},"min":{"1min":0.40300000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.67900000000000005,"5min":1.169,"15min":1.169},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.64400000000000002,"15min":0.64200000000000002},"min":{"1min":0.42199999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.88500000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.68400000000000005}]},{"osd":6,"last update":"Fri Jul 14 12:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.70799999999999996,"15min":0.71999999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.95999999999999996,"5min":1.21,"15min":1.21},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.73499999999999999,"15min":0.753},"min":{"1min":0.46100000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.79800000000000004,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":0.73299999999999998}]},{"osd":7,"last update":"Fri Jul 14 12:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.75700000000000001,"15min":0.754},"min":{"1min":0.56200000000000006,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.96299999999999997,"5min":1.327,"15min":1.327},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.76400000000000001,"15min":0.76300000000000001},"min":{"1min":0.57199999999999995,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.93100000000000005,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.82099999999999995}]},{"osd":8,"last update":"Fri Jul 14 12:00:46 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.64300000000000002}]},{"osd":9,"last update":"Fri Jul 14 12:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.81799999999999995,"15min":0.80700000000000005},"min":{"1min":0.52600000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":3.9089999999999998,"5min":3.9089999999999998,"15min":3.9089999999999998},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.81699999999999995,"15min":0.81499999999999995},"min":{"1min":0.63300000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.98299999999999998,"5min":1.385,"15min":1.385},"last":0.88800000000000001}]},{"osd":10,"last update":"Fri Jul 14 12:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.60799999999999998,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.995,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.79800000000000004,"15min":0.80300000000000005},"min":{"1min":0.59699999999999998,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.91600000000000004,"5min":2.1019999999999999,"15min":2.1019999999999999},"last":0.872}]}]},{"osd":10,"up_from":72,"seq":309237645344,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":1048,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1073152,"data_stored":1081734,"data_compressed":53729,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":3.375,"5min":3.375,"15min":3.375},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.67200000000000004}]},{"osd":1,"last update":"Fri Jul 14 12:00:37 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.496,"15min":0.504},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.63400000000000001,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.30199999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.65700000000000003}]},{"osd":2,"last update":"Fri Jul 14 12:00:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.60599999999999998,"15min":0.60699999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.61099999999999999,"15min":0.61199999999999999},"min":{"1min":0.42799999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.69199999999999995}]},{"osd":3,"last update":"Fri Jul 14 12:00:37 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.70399999999999996,"15min":0.67100000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":7.1159999999999997,"5min":7.1159999999999997,"15min":7.1159999999999997},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.67400000000000004,"15min":0.68300000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.80100000000000005,"5min":0.996,"15min":0.996},"last":0.56799999999999995}]},{"osd":4,"last update":"Fri Jul 14 12:00:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.56899999999999995,"15min":0.57499999999999996},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77400000000000002,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.38100000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.77400000000000002,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.77200000000000002}]},{"osd":5,"last update":"Fri Jul 14 12:00:37 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.65000000000000002,"15min":0.64800000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.749},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.70199999999999996,"15min":0.71399999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.79700000000000004,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.82499999999999996}]},{"osd":6,"last update":"Fri Jul 14 12:00:37 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.80600000000000005,"15min":0.78200000000000003},"min":{"1min":0.47999999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":7.0190000000000001,"5min":7.0190000000000001,"15min":7.0190000000000001},"last":0.88},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.82499999999999996,"15min":0.80500000000000005},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":5.5110000000000001,"5min":5.5110000000000001,"15min":5.5110000000000001},"last":0.76100000000000001}]},{"osd":7,"last update":"Fri Jul 14 11:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.60899999999999999}]},{"osd":8,"last update":"Fri Jul 14 12:00:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.77100000000000002,"15min":0.78400000000000003},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.76300000000000001,"15min":0.77000000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.96699999999999997,"5min":1.054,"15min":1.054},"last":0.95499999999999996}]},{"osd":9,"last update":"Fri Jul 14 12:00:37 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.80800000000000005,"15min":0.81999999999999995},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.90200000000000002,"5min":1.075,"15min":1.075},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.84399999999999997,"15min":0.82599999999999996},"min":{"1min":0.59399999999999997,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":5.367,"5min":5.367,"15min":5.367},"last":0.89600000000000002}]},{"osd":11,"last update":"Fri Jul 14 12:00:58 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.76400000000000001,"15min":0.77000000000000002},"min":{"1min":0.52100000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.86599999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.76800000000000002,"15min":0.77300000000000002},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.95199999999999996,"5min":1.05,"15min":1.05},"last":0.93700000000000006}]}]},{"osd":9,"up_from":67,"seq":287762808869,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":1007616,"data_stored":487337,"data_compressed":17563,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 12:00:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.63,"15min":0.64400000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":1.212},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.64100000000000001,"15min":0.65000000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.99299999999999999}]},{"osd":1,"last update":"Fri Jul 14 12:00:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.755,"15min":0.76800000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.877,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":1.1379999999999999},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.80200000000000005,"15min":0.81599999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.82899999999999996}]},{"osd":2,"last update":"Fri Jul 14 12:00:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.76800000000000002,"15min":0.77700000000000002},"min":{"1min":0.54400000000000004,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.99199999999999999,"5min":1.147,"15min":1.147},"last":1.198},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.79200000000000004,"15min":0.81200000000000006},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.91100000000000003,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.96999999999999997}]},{"osd":3,"last update":"Fri Jul 14 12:00:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.73599999999999999,"15min":0.75900000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.79700000000000004,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":1.173},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.75900000000000001,"15min":0.78000000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0980000000000001,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":1.0169999999999999}]},{"osd":4,"last update":"Fri Jul 14 12:00:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.871,"15min":0.89700000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.95599999999999996,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.94999999999999996,"15min":0.98199999999999998},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.012,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":1.056}]},{"osd":5,"last update":"Fri Jul 14 12:00:16 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.88,"15min":0.90200000000000002},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.94799999999999995,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":1.2290000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.93200000000000005,"15min":0.95599999999999996},"min":{"1min":0.58599999999999997,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0589999999999999,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":1.246}]},{"osd":6,"last update":"Fri Jul 14 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":1.006}]},{"osd":7,"last update":"Fri Jul 14 12:00:16 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.995,"15min":1.024},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.05,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.1060000000000001},{"interface":"front","average":{"1min":0.82399999999999995,"5min":1.008,"15min":1.0389999999999999},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.1240000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":1.083}]},{"osd":8,"last update":"Fri Jul 14 12:00:16 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.96899999999999997,"15min":0.98699999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.306,"5min":1.391,"15min":1.391},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.99299999999999999,"15min":1.0169999999999999},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.0800000000000001,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":1.3260000000000001}]},{"osd":10,"last update":"Fri Jul 14 12:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.94099999999999995,"15min":0.94699999999999995},"min":{"1min":0.627,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.2130000000000001,"5min":1.238,"15min":1.238},"last":1.806},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.93300000000000005,"15min":0.94099999999999995},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.1950000000000001,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":2.2989999999999999}]},{"osd":11,"last update":"Fri Jul 14 12:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":1.762},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":1.6120000000000001}]}]},{"osd":8,"up_from":59,"seq":253403070506,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":1003520,"data_stored":487715,"data_compressed":17563,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 12:00:51 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.67800000000000005,"15min":0.63400000000000001},"min":{"1min":0.434,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":8.0559999999999992,"5min":8.0559999999999992,"15min":8.0559999999999992},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.61299999999999999,"15min":0.59699999999999998},"min":{"1min":0.40100000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":4.1600000000000001,"5min":4.1600000000000001,"15min":4.1600000000000001},"last":0.437}]},{"osd":1,"last update":"Fri Jul 14 12:00:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.60499999999999998,"15min":0.58699999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.63700000000000001,"15min":0.63},"min":{"1min":0.53000000000000003,"5min":0.376,"15min":0.376},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.52100000000000002}]},{"osd":2,"last update":"Fri Jul 14 12:00:51 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.47999999999999998,"15min":0.47399999999999998},"min":{"1min":0.38,"5min":0.309,"15min":0.309},"max":{"1min":0.66000000000000003,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.51000000000000001,"15min":0.51800000000000002},"min":{"1min":0.30599999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.56999999999999995}]},{"osd":3,"last update":"Fri Jul 14 12:00:51 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":0.71399999999999997,"15min":0.67000000000000004},"min":{"1min":0.41099999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":8.6929999999999996,"5min":8.6929999999999996,"15min":8.6929999999999996},"last":0.64600000000000002},{"interface":"front","average":{"1min":1.0589999999999999,"5min":0.72999999999999998,"15min":0.68100000000000005},"min":{"1min":0.45300000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":8.2520000000000007,"5min":8.2520000000000007,"15min":8.2520000000000007},"last":0.55400000000000005}]},{"osd":4,"last update":"Fri Jul 14 12:00:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.622,"15min":0.622},"min":{"1min":0.40600000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81699999999999995,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.66900000000000004,"15min":0.65200000000000002},"min":{"1min":0.53600000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.60399999999999998}]},{"osd":5,"last update":"Fri Jul 14 12:00:51 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.61799999999999999,"15min":0.621},"min":{"1min":0.35399999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.83699999999999997,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.621},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.59499999999999997,"15min":0.59099999999999997},"min":{"1min":0.40100000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.58799999999999997}]},{"osd":6,"last update":"Fri Jul 14 12:00:51 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.747,"15min":0.751},"min":{"1min":0.56100000000000005,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.80400000000000005,"15min":0.79900000000000004},"min":{"1min":0.56399999999999995,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":2.7429999999999999,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":0.67300000000000004}]},{"osd":7,"last update":"Fri Jul 14 12:00:51 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.75800000000000001,"15min":0.755},"min":{"1min":0.47199999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.745,"15min":0.749},"min":{"1min":0.59399999999999997,"5min":0.435,"15min":0.435},"max":{"1min":1.0600000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.72399999999999998}]},{"osd":9,"last update":"Fri Jul 14 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.78500000000000003,"15min":0.77900000000000003},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.80400000000000005,"15min":0.79800000000000004},"min":{"1min":0.59299999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.0269999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.76400000000000001}]},{"osd":10,"last update":"Fri Jul 14 12:00:37 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.78700000000000003,"15min":0.78000000000000003},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.77800000000000002,"15min":0.76800000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.84999999999999998}]},{"osd":11,"last update":"Fri Jul 14 12:00:45 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.79400000000000004}]}]},{"osd":7,"up_from":52,"seq":223338299439,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":1048576,"data_stored":1079250,"data_compressed":53729,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.443,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.88300000000000001,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59699999999999998,"15min":0.59599999999999997},"min":{"1min":0.50700000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.495}]},{"osd":1,"last update":"Fri Jul 14 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42299999999999999,"15min":0.41599999999999998},"min":{"1min":0.317,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.69699999999999995,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44800000000000001},"min":{"1min":0.312,"5min":0.249,"15min":0.249},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.50900000000000001}]},{"osd":2,"last update":"Fri Jul 14 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59399999999999997,"15min":0.59699999999999998},"min":{"1min":0.39100000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73199999999999998,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.63700000000000001,"15min":0.621},"min":{"1min":0.51700000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.68500000000000005}]},{"osd":3,"last update":"Fri Jul 14 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.623,"15min":0.626},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85599999999999998,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.64300000000000002,"15min":0.65200000000000002},"min":{"1min":0.47299999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.75900000000000001,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.77700000000000002}]},{"osd":4,"last update":"Fri Jul 14 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52000000000000002,"15min":0.504},"min":{"1min":0.42699999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.71099999999999997,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.52300000000000002,"15min":0.52900000000000003},"min":{"1min":0.38300000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.80600000000000005,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.54600000000000004}]},{"osd":5,"last update":"Fri Jul 14 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.64400000000000002,"15min":0.63800000000000001},"min":{"1min":0.54800000000000004,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.79900000000000004,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71599999999999997,"15min":0.72099999999999997},"min":{"1min":0.54900000000000004,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.82399999999999995,"5min":1.341,"15min":1.341},"last":0.76200000000000001}]},{"osd":6,"last update":"Fri Jul 14 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.71199999999999997,"15min":0.71699999999999997},"min":{"1min":0.53300000000000003,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.81899999999999995,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.68500000000000005,"15min":0.69299999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.80600000000000005,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.85999999999999999}]},{"osd":8,"last update":"Fri Jul 14 12:00:51 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.72999999999999998,"15min":0.70599999999999996},"min":{"1min":0.46999999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.96399999999999997,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.71499999999999997,"15min":0.70399999999999996},"min":{"1min":0.54200000000000004,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93000000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.89000000000000001}]},{"osd":9,"last update":"Fri Jul 14 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.58199999999999996,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.93500000000000005,"5min":0.995,"15min":0.995},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.77600000000000002,"15min":0.77900000000000003},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.83499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.91900000000000004}]},{"osd":10,"last update":"Fri Jul 14 12:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67800000000000005,"15min":0.67900000000000005},"min":{"1min":0.46700000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.75600000000000001,"15min":0.76200000000000001},"min":{"1min":0.51000000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.90400000000000003,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.73099999999999998}]},{"osd":11,"last update":"Fri Jul 14 12:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.878}]}]},{"osd":6,"up_from":47,"seq":201863462962,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":987136,"data_stored":486889,"data_compressed":17563,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 12:00:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.48699999999999999,"15min":0.47299999999999998},"min":{"1min":0.39100000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.60599999999999998,"15min":0.59799999999999998},"min":{"1min":0.46600000000000003,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.71499999999999997}]},{"osd":1,"last update":"Fri Jul 14 12:00:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.65600000000000003,"15min":0.64600000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.72699999999999998,"15min":0.72199999999999998},"min":{"1min":0.51200000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.93300000000000005}]},{"osd":2,"last update":"Fri Jul 14 12:00:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.65500000000000003,"15min":0.64500000000000002},"min":{"1min":0.48999999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.93799999999999994,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.70099999999999996,"15min":0.68999999999999995},"min":{"1min":0.41399999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0049999999999999,"5min":1.155,"15min":1.155},"last":0.84499999999999997}]},{"osd":3,"last update":"Fri Jul 14 12:00:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.61399999999999999,"15min":0.59999999999999998},"min":{"1min":0.46200000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.93600000000000005,"5min":1.107,"15min":1.107},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.70899999999999996,"15min":0.69699999999999995},"min":{"1min":0.57499999999999996,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91500000000000004,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.82199999999999995}]},{"osd":4,"last update":"Fri Jul 14 12:00:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.68500000000000005,"15min":0.67300000000000004},"min":{"1min":0.441,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.754,"15min":0.73999999999999999},"min":{"1min":0.57199999999999995,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.92100000000000004}]},{"osd":5,"last update":"Fri Jul 14 12:00:06 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.79900000000000004,"15min":0.79300000000000004},"min":{"1min":0.67600000000000005,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1000000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.745,"15min":0.73399999999999999},"min":{"1min":0.55000000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.749}]},{"osd":7,"last update":"Fri Jul 14 12:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.80100000000000005,"15min":0.76200000000000001},"min":{"1min":0.59299999999999997,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":1.0920000000000001,"5min":0.81299999999999994,"15min":0.74099999999999999},"min":{"1min":0.57399999999999995,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":6.4400000000000004,"5min":6.4400000000000004,"15min":6.4400000000000004},"last":0.99299999999999999}]},{"osd":8,"last update":"Fri Jul 14 12:00:48 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.83399999999999996,"15min":0.81100000000000005},"min":{"1min":0.68500000000000005,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":1.012},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.85099999999999998,"15min":0.82199999999999995},"min":{"1min":0.67200000000000004,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.79700000000000004}]},{"osd":9,"last update":"Fri Jul 14 12:00:13 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.93600000000000005,"15min":0.93200000000000005},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.92900000000000005,"15min":0.92200000000000004},"min":{"1min":0.68000000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.96499999999999997}]},{"osd":10,"last update":"Fri Jul 14 12:00:37 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.01,"5min":1.012,"15min":1.012},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":1.054}]},{"osd":11,"last update":"Fri Jul 14 12:00:48 2023","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.046,"15min":1.046},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":1.0369999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":995328,"data_stored":491713,"data_compressed":17563,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54800000000000004,"15min":0.52800000000000002},"min":{"1min":0.30199999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.83299999999999996,"5min":1.153,"15min":1.153},"last":2.4860000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.60799999999999998,"15min":0.72499999999999998},"min":{"1min":0.35099999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.872,"5min":5.0940000000000003,"15min":5.0940000000000003},"last":2.4140000000000001}]},{"osd":2,"last update":"Fri Jul 14 12:00:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.55500000000000005,"15min":0.50700000000000001},"min":{"1min":0.32600000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.82099999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.624,"15min":0.69699999999999995},"min":{"1min":0.434,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.84899999999999998,"5min":5.0570000000000004,"15min":5.0570000000000004},"last":0.57699999999999996}]},{"osd":3,"last update":"Fri Jul 14 12:00:00 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60699999999999998,"15min":0.59699999999999998},"min":{"1min":0.39900000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.95999999999999996,"5min":1.054,"15min":1.054},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.42399999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.92600000000000005,"5min":1.006,"15min":1.006},"last":0.67500000000000004}]},{"osd":4,"last update":"Fri Jul 14 12:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.45800000000000002,"15min":0.42799999999999999},"min":{"1min":0.36799999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.69099999999999995,"5min":1,"15min":1},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.48099999999999998,"15min":0.45800000000000002},"min":{"1min":0.33700000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.38200000000000001}]},{"osd":5,"last update":"Fri Jul 14 12:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.64200000000000002,"15min":0.59599999999999997},"min":{"1min":0.48799999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.61499999999999999,"15min":0.56699999999999995},"min":{"1min":0.53300000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.94999999999999996,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.621}]},{"osd":6,"last update":"Fri Jul 14 12:00:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.65700000000000003,"15min":0.63900000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.749,"5min":0.69399999999999995,"15min":0.67500000000000004},"min":{"1min":0.53800000000000003,"5min":0.44,"15min":0.44},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":2.395}]},{"osd":7,"last update":"Fri Jul 14 12:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.55600000000000005,"15min":0.53600000000000003},"min":{"1min":0.42699999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.64600000000000002,"15min":0.60399999999999998},"min":{"1min":0.47899999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.55200000000000005}]},{"osd":8,"last update":"Fri Jul 14 12:00:53 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.73699999999999999,"15min":0.70899999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.071,"5min":1.105,"15min":1.105},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.70199999999999996,"15min":0.68999999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.59399999999999997}]},{"osd":9,"last update":"Fri Jul 14 12:00:18 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.81299999999999994,"15min":0.81100000000000005},"min":{"1min":0.60399999999999998,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.038,"5min":1.2,"15min":1.2},"last":2.5169999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.77200000000000002,"15min":0.77500000000000002},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.026,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":2.4260000000000002}]},{"osd":10,"last update":"Fri Jul 14 12:00:38 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.71099999999999997,"15min":0.70699999999999996},"min":{"1min":0.55100000000000005,"5min":0.496,"15min":0.496},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.84899999999999998,"15min":0.85899999999999999},"min":{"1min":0.61599999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0580000000000001,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":0.66000000000000003}]},{"osd":11,"last update":"Fri Jul 14 12:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.70999999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738446,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":1011712,"data_stored":488345,"data_compressed":17563,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 12:00:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.58899999999999997,"15min":0.55000000000000004},"min":{"1min":0.33600000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.96999999999999997,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.5},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.57699999999999996,"15min":0.53700000000000003},"min":{"1min":0.43099999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.92800000000000005,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.58999999999999997}]},{"osd":2,"last update":"Fri Jul 14 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.64400000000000002,"15min":0.59599999999999997},"min":{"1min":0.435,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.61599999999999999,"15min":0.56899999999999995},"min":{"1min":0.38200000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.89900000000000002,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.627}]},{"osd":3,"last update":"Fri Jul 14 11:58:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.54500000000000004,"15min":0.52300000000000002},"min":{"1min":0.313,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.55700000000000005,"15min":0.51100000000000001},"min":{"1min":0.39600000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.375}]},{"osd":4,"last update":"Fri Jul 14 12:00:23 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.623,"15min":0.58699999999999997},"min":{"1min":0.38500000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.64700000000000002,"15min":0.59399999999999997},"min":{"1min":0.45100000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.93200000000000005,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.60099999999999998}]},{"osd":5,"last update":"Fri Jul 14 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.68899999999999995,"15min":0.65600000000000003},"min":{"1min":0.45200000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.71599999999999997,"15min":0.68300000000000005},"min":{"1min":0.57299999999999995,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.96199999999999997,"5min":1.131,"15min":1.131},"last":0.78500000000000003}]},{"osd":6,"last update":"Fri Jul 14 12:00:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.67600000000000005,"15min":0.66000000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.92600000000000005,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.435},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.68700000000000006,"15min":0.68100000000000005},"min":{"1min":0.40999999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86899999999999999,"5min":1.081,"15min":1.081},"last":0.56499999999999995}]},{"osd":7,"last update":"Fri Jul 14 12:00:48 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.54400000000000004}]},{"osd":8,"last update":"Fri Jul 14 12:00:52 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.80700000000000005,"15min":0.80300000000000005},"min":{"1min":0.54200000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":3.089,"5min":3.089,"15min":3.089},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.876,"5min":0.82699999999999996,"15min":0.81999999999999995},"min":{"1min":0.67000000000000004,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.83299999999999996}]},{"osd":9,"last update":"Fri Jul 14 12:00:13 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.85799999999999998,"15min":0.86299999999999999},"min":{"1min":0.59699999999999998,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.747},{"interface":"front","average":{"1min":0.871,"5min":0.90700000000000003,"15min":0.91300000000000003},"min":{"1min":0.57899999999999996,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.204,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.79200000000000004}]},{"osd":10,"last update":"Fri Jul 14 12:00:35 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.92400000000000004,"15min":0.93200000000000005},"min":{"1min":0.66800000000000004,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.083,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.93100000000000005,"15min":0.94599999999999995},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.232,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.70999999999999996}]},{"osd":11,"last update":"Fri Jul 14 12:00:57 2023","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":0.86099999999999999,"15min":0.82699999999999996},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":4.4050000000000002,"5min":4.4050000000000002,"15min":4.4050000000000002},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.0720000000000001,"5min":0.90900000000000003,"15min":0.88200000000000001},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":4.3869999999999996,"5min":4.3869999999999996,"15min":4.3869999999999996},"last":0.80700000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411398,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":1015808,"data_stored":491885,"data_compressed":17563,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 12:00:40 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.56200000000000006,"15min":0.504},"min":{"1min":0.373,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.78200000000000003,"5min":6.54,"15min":6.54},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.54200000000000004,"15min":0.504},"min":{"1min":0.40600000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.379}]},{"osd":1,"last update":"Fri Jul 14 12:00:40 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.56100000000000005,"15min":0.505},"min":{"1min":0.34000000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.89000000000000001,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.56699999999999995,"15min":0.51100000000000001},"min":{"1min":0.34799999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.86199999999999999,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.61799999999999999}]},{"osd":3,"last update":"Fri Jul 14 12:00:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.58499999999999996,"15min":0.54600000000000004},"min":{"1min":0.39200000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.83299999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.436},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.59599999999999997,"15min":0.55900000000000005},"min":{"1min":0.41599999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.88400000000000001,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.49199999999999999}]},{"osd":4,"last update":"Fri Jul 14 12:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.60099999999999998}]},{"osd":5,"last update":"Fri Jul 14 12:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.47199999999999998,"15min":0.44400000000000001},"min":{"1min":0.28899999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.498,"15min":0.46100000000000002},"min":{"1min":0.318,"5min":0.252,"15min":0.252},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.57799999999999996}]},{"osd":6,"last update":"Fri Jul 14 12:00:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.65200000000000002,"15min":0.63700000000000001},"min":{"1min":0.50700000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.64800000000000002,"15min":0.63800000000000001},"min":{"1min":0.46600000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.52300000000000002}]},{"osd":7,"last update":"Fri Jul 14 12:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.74399999999999999}]},{"osd":8,"last update":"Fri Jul 14 12:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.61099999999999999,"15min":0.60399999999999998},"min":{"1min":0.35799999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.63900000000000001,"15min":0.627},"min":{"1min":0.41399999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.75700000000000001}]},{"osd":9,"last update":"Fri Jul 14 12:00:18 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.752,"15min":0.74199999999999999},"min":{"1min":0.46200000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.76500000000000001,"15min":0.753},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.72699999999999998}]},{"osd":10,"last update":"Fri Jul 14 12:00:40 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.76500000000000001,"15min":0.753},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.749,"15min":0.73599999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.79000000000000004}]},{"osd":11,"last update":"Fri Jul 14 12:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.83799999999999997}]}]},{"osd":3,"up_from":26,"seq":111669149761,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":999424,"data_stored":486645,"data_compressed":17563,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 12:00:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.498,"15min":0.40000000000000002},"min":{"1min":0.36299999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":2.7080000000000002,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.53600000000000003,"15min":0.42899999999999999},"min":{"1min":0.379,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":5.1790000000000003,"5min":5.1790000000000003,"15min":5.1790000000000003},"last":0.55600000000000005}]},{"osd":1,"last update":"Fri Jul 14 12:00:57 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.71499999999999997,"15min":0.59699999999999998},"min":{"1min":0.53200000000000003,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":2.7930000000000001,"5min":2.7930000000000001,"15min":2.7930000000000001},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.746,"5min":0.629,"15min":0.56100000000000005},"min":{"1min":0.55200000000000005,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.63700000000000001}]},{"osd":2,"last update":"Fri Jul 14 12:00:57 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.73699999999999999,"15min":0.628},"min":{"1min":0.59699999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":4.9610000000000003,"5min":4.9610000000000003,"15min":4.9610000000000003},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.73699999999999999,"15min":0.63400000000000001},"min":{"1min":0.53700000000000003,"5min":0.374,"15min":0.374},"max":{"1min":2.9249999999999998,"5min":2.9249999999999998,"15min":2.9249999999999998},"last":0.70899999999999996}]},{"osd":4,"last update":"Fri Jul 14 12:00:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.65800000000000003,"15min":0.61099999999999999},"min":{"1min":0.51200000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.871,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.622},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.72899999999999998,"15min":0.63700000000000001},"min":{"1min":0.56499999999999995,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":3.9100000000000001,"5min":3.9100000000000001,"15min":3.9100000000000001},"last":0.80800000000000005}]},{"osd":5,"last update":"Fri Jul 14 12:00:38 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.66900000000000004,"15min":0.63300000000000001},"min":{"1min":0.42499999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.77700000000000002,"15min":0.70299999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":5.1440000000000001,"5min":5.1440000000000001,"15min":5.1440000000000001},"last":0.73999999999999999}]},{"osd":6,"last update":"Fri Jul 14 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.749,"15min":0.71799999999999997},"min":{"1min":0.53300000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.97999999999999998,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":1.1890000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.69499999999999995,"15min":0.69399999999999995},"min":{"1min":0.44700000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.95699999999999996,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":1.036}]},{"osd":7,"last update":"Fri Jul 14 12:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.878,"15min":0.872},"min":{"1min":0.55500000000000005,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":3.931,"5min":3.931,"15min":3.931},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.89800000000000002,"15min":0.89000000000000001},"min":{"1min":0.66300000000000003,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.242,"5min":1.383,"15min":1.383},"last":1.107}]},{"osd":8,"last update":"Fri Jul 14 11:58:46 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":1.0009999999999999}]},{"osd":9,"last update":"Fri Jul 14 12:00:17 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":1.0069999999999999,"15min":1.0249999999999999},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.236,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":1.153},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.84899999999999998,"15min":0.86699999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0369999999999999,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.94799999999999995}]},{"osd":10,"last update":"Fri Jul 14 12:00:35 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.89100000000000001,"15min":0.88500000000000001},"min":{"1min":0.68200000000000005,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.863,"5min":1.863,"15min":1.863},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.0800000000000001,"5min":0.97099999999999997,"15min":0.95299999999999996},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":5.0140000000000002,"5min":5.0140000000000002,"15min":5.0140000000000002},"last":1.0509999999999999}]},{"osd":11,"last update":"Fri Jul 14 12:00:46 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.9980000000000002,"5min":2.9980000000000002,"15min":2.9980000000000002},"last":1.232},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.9379999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.1359999999999999}]}]},{"osd":4,"up_from":34,"seq":146028888124,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":991232,"data_stored":486867,"data_compressed":17563,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.52700000000000002,"15min":0.504},"min":{"1min":0.41399999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.53700000000000003,"15min":0.52300000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.80000000000000004,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.48299999999999998}]},{"osd":1,"last update":"Fri Jul 14 12:00:22 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.373,"15min":0.35499999999999998},"min":{"1min":0.29599999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.56899999999999995,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.38600000000000001,"15min":0.375},"min":{"1min":0.27000000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.55900000000000005,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.59299999999999997}]},{"osd":2,"last update":"Fri Jul 14 12:00:22 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.56100000000000005,"15min":0.54300000000000004},"min":{"1min":0.41099999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.59199999999999997,"15min":0.55700000000000005},"min":{"1min":0.432,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.83799999999999997,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.747}]},{"osd":3,"last update":"Fri Jul 14 12:00:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57399999999999995,"15min":0.55200000000000005},"min":{"1min":0.35299999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.79600000000000004,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.58499999999999996,"15min":0.56699999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.89400000000000002}]},{"osd":5,"last update":"Fri Jul 14 12:00:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.60999999999999999,"15min":0.58499999999999996},"min":{"1min":0.45200000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.80600000000000005,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.60099999999999998,"15min":0.58899999999999997},"min":{"1min":0.39200000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.67100000000000004}]},{"osd":6,"last update":"Fri Jul 14 12:00:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.623,"15min":0.61599999999999999},"min":{"1min":0.46800000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.80800000000000005,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.69199999999999995,"15min":0.68400000000000005},"min":{"1min":0.48299999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.90900000000000003}]},{"osd":7,"last update":"Fri Jul 14 12:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56599999999999995,"15min":0.54700000000000004},"min":{"1min":0.34499999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.78300000000000003,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53900000000000003,"15min":0.51900000000000002},"min":{"1min":0.40600000000000003,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.71099999999999997,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.76400000000000001}]},{"osd":8,"last update":"Fri Jul 14 12:00:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.63500000000000001,"15min":0.621},"min":{"1min":0.44900000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.73399999999999999,"15min":0.72399999999999998},"min":{"1min":0.52000000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.94499999999999995,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":1.0169999999999999}]},{"osd":9,"last update":"Fri Jul 14 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71099999999999997,"15min":0.70999999999999996},"min":{"1min":0.55600000000000005,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":1.03},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.80000000000000004,"15min":0.80600000000000005},"min":{"1min":0.55900000000000005,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.92800000000000005,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.98799999999999999}]},{"osd":10,"last update":"Fri Jul 14 12:00:39 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.77100000000000002,"15min":0.77300000000000002},"min":{"1min":0.53900000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.91100000000000003,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.68899999999999995,"15min":0.69099999999999995},"min":{"1min":0.51800000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":1.002}]},{"osd":11,"last update":"Fri Jul 14 12:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":1.0429999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":1.056}]}]},{"osd":5,"up_from":39,"seq":167503724600,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":1036,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":1060864,"data_stored":1079024,"data_compressed":53729,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 12:00:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.60199999999999998,"15min":0.60399999999999998},"min":{"1min":0.46600000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77500000000000002,"5min":1.018,"15min":1.018},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59799999999999998,"15min":0.60899999999999999},"min":{"1min":0.435,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.94399999999999995,"5min":1.127,"15min":1.127},"last":0.73499999999999999}]},{"osd":1,"last update":"Fri Jul 14 12:00:39 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.57799999999999996,"15min":0.53600000000000003},"min":{"1min":0.497,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.85599999999999998,"5min":1.018,"15min":1.018},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.58499999999999996,"15min":0.54500000000000004},"min":{"1min":0.48999999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.68400000000000005}]},{"osd":2,"last update":"Fri Jul 14 12:00:39 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.40600000000000003,"15min":0.38400000000000001},"min":{"1min":0.37,"5min":0.22,"15min":0.22},"max":{"1min":0.67100000000000004,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.42599999999999999,"15min":0.41299999999999998},"min":{"1min":0.39300000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.59499999999999997,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.42599999999999999}]},{"osd":3,"last update":"Fri Jul 14 12:00:39 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65700000000000003,"15min":0.64500000000000002},"min":{"1min":0.436,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.88800000000000001,"5min":3.6640000000000001,"15min":3.6640000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.66700000000000004,"15min":0.66000000000000003},"min":{"1min":0.52600000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.58599999999999997}]},{"osd":4,"last update":"Fri Jul 14 12:00:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.59199999999999997,"15min":0.56100000000000005},"min":{"1min":0.5,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.623},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.59799999999999998,"15min":0.57299999999999995},"min":{"1min":0.51000000000000001,"5min":0.379,"15min":0.379},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.77100000000000002}]},{"osd":6,"last update":"Fri Jul 14 12:00:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.67400000000000004,"15min":0.67000000000000004},"min":{"1min":0.46800000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.91800000000000004,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.69699999999999995,"15min":0.69599999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.56000000000000005}]},{"osd":7,"last update":"Fri Jul 14 12:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.67300000000000004,"15min":0.66700000000000004},"min":{"1min":0.56200000000000006,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.73399999999999999,"15min":0.745},"min":{"1min":0.55600000000000005,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.095,"5min":3.6440000000000001,"15min":3.6440000000000001},"last":0.755}]},{"osd":8,"last update":"Fri Jul 14 12:00:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.59799999999999998,"15min":0.57799999999999996},"min":{"1min":0.41699999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.879,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.58699999999999997,"15min":0.58099999999999996},"min":{"1min":0.442,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.66800000000000004}]},{"osd":9,"last update":"Fri Jul 14 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.79000000000000004,"15min":0.78400000000000003},"min":{"1min":0.59499999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.77600000000000002,"15min":0.77000000000000002},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.85999999999999999}]},{"osd":10,"last update":"Fri Jul 14 12:00:34 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.68799999999999994,"15min":0.67200000000000004},"min":{"1min":0.63,"5min":0.504,"15min":0.504},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.75600000000000001,"15min":0.74199999999999999},"min":{"1min":0.59199999999999997,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.83199999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36166,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36166,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36166,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36166,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36166,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":5482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T12:01:05.980 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph pg dump --format=json 2023-07-14T12:01:06.715 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T12:01:07.061 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:06 smithi017 ceph-mon[126339]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T12:01:07.061 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:06 smithi017 ceph-mon[126339]: pgmap v320: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-14T12:01:07.061 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:06 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2023-07-14T12:01:07.061 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:06 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2023-07-14T12:01:07.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:06 smithi093 ceph-mon[128922]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T12:01:07.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:06 smithi093 ceph-mon[128922]: pgmap v320: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-14T12:01:07.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:06 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2023-07-14T12:01:07.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:06 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2023-07-14T12:01:07.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:06 smithi097 ceph-mon[129313]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T12:01:07.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:06 smithi097 ceph-mon[129313]: pgmap v320: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-14T12:01:07.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:06 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2023-07-14T12:01:07.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:06 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2023-07-14T12:01:08.051 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:07 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2023-07-14T12:01:08.051 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:07 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2023-07-14T12:01:08.052 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:07 smithi017 ceph-mon[126339]: osdmap e91: 12 total, 12 up, 12 in 2023-07-14T12:01:08.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:07 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2023-07-14T12:01:08.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:07 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2023-07-14T12:01:08.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:07 smithi093 ceph-mon[128922]: osdmap e91: 12 total, 12 up, 12 in 2023-07-14T12:01:08.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:07 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2023-07-14T12:01:08.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:07 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2023-07-14T12:01:08.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:07 smithi097 ceph-mon[129313]: osdmap e91: 12 total, 12 up, 12 in 2023-07-14T12:01:08.263 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.283+0000 7f9d4b631700 1 -- 172.21.15.17:0/2418719135 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d44104ea0 msgr2=0x7f9d4410ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:08.264 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.283+0000 7f9d4b631700 1 --2- 172.21.15.17:0/2418719135 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d44104ea0 0x7f9d4410ca20 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c00b0d0 tx=0x7f9d3c00b3e0 comp rx=0 tx=0).stop 2023-07-14T12:01:08.264 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.283+0000 7f9d4b631700 1 -- 172.21.15.17:0/2418719135 shutdown_connections 2023-07-14T12:01:08.264 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.283+0000 7f9d4b631700 1 --2- 172.21.15.17:0/2418719135 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d440ffe00 0x7f9d441001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:08.264 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.283+0000 7f9d4b631700 1 --2- 172.21.15.17:0/2418719135 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d44104ea0 0x7f9d4410ca20 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:08.264 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.283+0000 7f9d4b631700 1 --2- 172.21.15.17:0/2418719135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d441007a0 0x7f9d44104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:08.264 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.283+0000 7f9d4b631700 1 -- 172.21.15.17:0/2418719135 >> 172.21.15.17:0/2418719135 conn(0x7f9d440fb660 msgr2=0x7f9d440fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:08.265 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.283+0000 7f9d4b631700 1 -- 172.21.15.17:0/2418719135 shutdown_connections 2023-07-14T12:01:08.265 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.283+0000 7f9d4b631700 1 -- 172.21.15.17:0/2418719135 wait complete. 2023-07-14T12:01:08.265 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.284+0000 7f9d4b631700 1 Processor -- start 2023-07-14T12:01:08.270 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.284+0000 7f9d4b631700 1 -- start start 2023-07-14T12:01:08.271 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.285+0000 7f9d4b631700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d440ffe00 0x7f9d44197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:08.271 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.285+0000 7f9d4b631700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d441007a0 0x7f9d441984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:08.271 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.285+0000 7f9d4b631700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d44104ea0 0x7f9d4419c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:08.271 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.285+0000 7f9d4b631700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4419cf40 con 0x7f9d44104ea0 2023-07-14T12:01:08.271 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.285+0000 7f9d4b631700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4419d0b0 con 0x7f9d440ffe00 2023-07-14T12:01:08.271 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.285+0000 7f9d4b631700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4419d270 con 0x7f9d441007a0 2023-07-14T12:01:08.271 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.285+0000 7f9d493cd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d440ffe00 0x7f9d44197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:08.272 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.285+0000 7f9d49bce700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d44104ea0 0x7f9d4419c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:08.272 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.285+0000 7f9d48bcc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d441007a0 0x7f9d441984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:08.272 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.285+0000 7f9d493cd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d440ffe00 0x7f9d44197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.17:33424/0 (socket says 172.21.15.17:33424) 2023-07-14T12:01:08.272 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.285+0000 7f9d493cd700 1 -- 172.21.15.17:0/1599690315 learned_addr learned my addr 172.21.15.17:0/1599690315 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:08.272 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.285+0000 7f9d49bce700 1 -- 172.21.15.17:0/1599690315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d441007a0 msgr2=0x7f9d441984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:08.272 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.286+0000 7f9d49bce700 1 --2- 172.21.15.17:0/1599690315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d441007a0 0x7f9d441984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:08.272 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.286+0000 7f9d49bce700 1 -- 172.21.15.17:0/1599690315 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d440ffe00 msgr2=0x7f9d44197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:08.272 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.286+0000 7f9d49bce700 1 --2- 172.21.15.17:0/1599690315 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d440ffe00 0x7f9d44197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:08.273 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.286+0000 7f9d49bce700 1 -- 172.21.15.17:0/1599690315 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d3c009d00 con 0x7f9d44104ea0 2023-07-14T12:01:08.273 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.286+0000 7f9d493cd700 1 --2- 172.21.15.17:0/1599690315 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d440ffe00 0x7f9d44197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T12:01:08.273 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.286+0000 7f9d49bce700 1 --2- 172.21.15.17:0/1599690315 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d44104ea0 0x7f9d4419c930 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c009fd0 tx=0x7f9d3c0077f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:08.273 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.286+0000 7f9d3a7fc700 1 -- 172.21.15.17:0/1599690315 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c00e040 con 0x7f9d44104ea0 2023-07-14T12:01:08.273 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.286+0000 7f9d3a7fc700 1 -- 172.21.15.17:0/1599690315 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d3c00bcb0 con 0x7f9d44104ea0 2023-07-14T12:01:08.273 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.286+0000 7f9d4b631700 1 -- 172.21.15.17:0/1599690315 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d4419d4f0 con 0x7f9d44104ea0 2023-07-14T12:01:08.273 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.286+0000 7f9d3a7fc700 1 -- 172.21.15.17:0/1599690315 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c005840 con 0x7f9d44104ea0 2023-07-14T12:01:08.273 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.286+0000 7f9d4b631700 1 -- 172.21.15.17:0/1599690315 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d441a4db0 con 0x7f9d44104ea0 2023-07-14T12:01:08.274 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.287+0000 7f9d4b631700 1 -- 172.21.15.17:0/1599690315 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d4404e210 con 0x7f9d44104ea0 2023-07-14T12:01:08.274 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.289+0000 7f9d3a7fc700 1 -- 172.21.15.17:0/1599690315 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9d3c004a50 con 0x7f9d44104ea0 2023-07-14T12:01:08.274 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.291+0000 7f9d3a7fc700 1 --2- 172.21.15.17:0/1599690315 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f9d300a11d0 0x7f9d300a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:08.275 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.291+0000 7f9d3a7fc700 1 -- 172.21.15.17:0/1599690315 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f9d3c0cdce0 con 0x7f9d44104ea0 2023-07-14T12:01:08.275 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.291+0000 7f9d493cd700 1 --2- 172.21.15.17:0/1599690315 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f9d300a11d0 0x7f9d300a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:08.275 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.292+0000 7f9d493cd700 1 --2- 172.21.15.17:0/1599690315 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f9d300a11d0 0x7f9d300a3680 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9d40011550 tx=0x7f9d40009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:08.275 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.295+0000 7f9d3a7fc700 1 -- 172.21.15.17:0/1599690315 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9d3c097110 con 0x7f9d44104ea0 2023-07-14T12:01:08.478 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.498+0000 7f9d4b631700 1 -- 172.21.15.17:0/1599690315 --> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9d440fc750 con 0x7f9d300a11d0 2023-07-14T12:01:08.491 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.511+0000 7f9d3a7fc700 1 -- 172.21.15.17:0/1599690315 <== mgr.14152 v2:172.21.15.17:6800/4237449858 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427578 (secure 0 0 0) 0x7f9d440fc750 con 0x7f9d300a11d0 2023-07-14T12:01:08.496 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T12:01:08.499 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.516+0000 7f9d4b631700 1 -- 172.21.15.17:0/1599690315 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f9d300a11d0 msgr2=0x7f9d300a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:08.499 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.516+0000 7f9d4b631700 1 --2- 172.21.15.17:0/1599690315 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f9d300a11d0 0x7f9d300a3680 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9d40011550 tx=0x7f9d40009260 comp rx=0 tx=0).stop 2023-07-14T12:01:08.499 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.516+0000 7f9d4b631700 1 -- 172.21.15.17:0/1599690315 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d44104ea0 msgr2=0x7f9d4419c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:08.499 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.516+0000 7f9d4b631700 1 --2- 172.21.15.17:0/1599690315 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d44104ea0 0x7f9d4419c930 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c009fd0 tx=0x7f9d3c0077f0 comp rx=0 tx=0).stop 2023-07-14T12:01:08.499 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.517+0000 7f9d4b631700 1 -- 172.21.15.17:0/1599690315 shutdown_connections 2023-07-14T12:01:08.499 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.517+0000 7f9d4b631700 1 --2- 172.21.15.17:0/1599690315 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f9d300a11d0 0x7f9d300a3680 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:08.499 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.517+0000 7f9d4b631700 1 --2- 172.21.15.17:0/1599690315 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d440ffe00 0x7f9d44197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:08.500 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.517+0000 7f9d4b631700 1 --2- 172.21.15.17:0/1599690315 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d44104ea0 0x7f9d4419c930 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:08.500 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.517+0000 7f9d4b631700 1 --2- 172.21.15.17:0/1599690315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d441007a0 0x7f9d441984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:08.500 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.517+0000 7f9d4b631700 1 -- 172.21.15.17:0/1599690315 >> 172.21.15.17:0/1599690315 conn(0x7f9d440fb660 msgr2=0x7f9d440fc040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:08.500 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.517+0000 7f9d4b631700 1 -- 172.21.15.17:0/1599690315 shutdown_connections 2023-07-14T12:01:08.500 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:08.517+0000 7f9d4b631700 1 -- 172.21.15.17:0/1599690315 wait complete. 2023-07-14T12:01:08.500 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2023-07-14T12:01:09.008 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2023-07-14T12:01:07.811382+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":600392,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":649,"num_read_kb":395,"num_write":390,"num_write_kb":3005,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":320,"ondisk_log_size":320,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335316,"kb_used_data":11892,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606508,"statfs":{"total":1151940427776,"available":1151597064192,"internally_reserved":0,"allocated":12177408,"data_stored":7634763,"data_compressed":319254,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.065757"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.813006+0000","last_change":"2023-07-14T11:59:43.468306+0000","last_active":"2023-07-14T12:00:06.813006+0000","last_peered":"2023-07-14T12:00:06.813006+0000","last_clean":"2023-07-14T12:00:06.813006+0000","last_became_active":"2023-07-14T11:59:43.467506+0000","last_became_peered":"2023-07-14T11:59:43.467506+0000","last_unstale":"2023-07-14T12:00:06.813006+0000","last_undegraded":"2023-07-14T12:00:06.813006+0000","last_fullsized":"2023-07-14T12:00:06.813006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:49:26.343044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.814808+0000","last_change":"2023-07-14T11:59:43.462562+0000","last_active":"2023-07-14T12:00:06.814808+0000","last_peered":"2023-07-14T12:00:06.814808+0000","last_clean":"2023-07-14T12:00:06.814808+0000","last_became_active":"2023-07-14T11:59:43.462126+0000","last_became_peered":"2023-07-14T11:59:43.462126+0000","last_unstale":"2023-07-14T12:00:06.814808+0000","last_undegraded":"2023-07-14T12:00:06.814808+0000","last_fullsized":"2023-07-14T12:00:06.814808+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:20:10.445422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.813957+0000","last_change":"2023-07-14T11:59:43.470205+0000","last_active":"2023-07-14T12:00:06.813957+0000","last_peered":"2023-07-14T12:00:06.813957+0000","last_clean":"2023-07-14T12:00:06.813957+0000","last_became_active":"2023-07-14T11:59:43.469804+0000","last_became_peered":"2023-07-14T11:59:43.469804+0000","last_unstale":"2023-07-14T12:00:06.813957+0000","last_undegraded":"2023-07-14T12:00:06.813957+0000","last_fullsized":"2023-07-14T12:00:06.813957+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:06:21.532674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.807812+0000","last_change":"2023-07-14T11:59:43.464834+0000","last_active":"2023-07-14T12:00:06.807812+0000","last_peered":"2023-07-14T12:00:06.807812+0000","last_clean":"2023-07-14T12:00:06.807812+0000","last_became_active":"2023-07-14T11:59:43.464252+0000","last_became_peered":"2023-07-14T11:59:43.464252+0000","last_unstale":"2023-07-14T12:00:06.807812+0000","last_undegraded":"2023-07-14T12:00:06.807812+0000","last_fullsized":"2023-07-14T12:00:06.807812+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:30:26.528049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808013+0000","last_change":"2023-07-14T11:59:43.460468+0000","last_active":"2023-07-14T12:00:06.808013+0000","last_peered":"2023-07-14T12:00:06.808013+0000","last_clean":"2023-07-14T12:00:06.808013+0000","last_became_active":"2023-07-14T11:59:43.460050+0000","last_became_peered":"2023-07-14T11:59:43.460050+0000","last_unstale":"2023-07-14T12:00:06.808013+0000","last_undegraded":"2023-07-14T12:00:06.808013+0000","last_fullsized":"2023-07-14T12:00:06.808013+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:43:21.915056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808330+0000","last_change":"2023-07-14T11:59:43.458605+0000","last_active":"2023-07-14T12:00:06.808330+0000","last_peered":"2023-07-14T12:00:06.808330+0000","last_clean":"2023-07-14T12:00:06.808330+0000","last_became_active":"2023-07-14T11:59:43.458233+0000","last_became_peered":"2023-07-14T11:59:43.458233+0000","last_unstale":"2023-07-14T12:00:06.808330+0000","last_undegraded":"2023-07-14T12:00:06.808330+0000","last_fullsized":"2023-07-14T12:00:06.808330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:27:55.968254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.328508+0000","last_change":"2023-07-14T11:59:43.465950+0000","last_active":"2023-07-14T12:00:07.328508+0000","last_peered":"2023-07-14T12:00:07.328508+0000","last_clean":"2023-07-14T12:00:07.328508+0000","last_became_active":"2023-07-14T11:59:43.465652+0000","last_became_peered":"2023-07-14T11:59:43.465652+0000","last_unstale":"2023-07-14T12:00:07.328508+0000","last_undegraded":"2023-07-14T12:00:07.328508+0000","last_fullsized":"2023-07-14T12:00:07.328508+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:48:42.466464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-14T12:01:06.807217+0000","last_change":"2023-07-14T12:00:06.812609+0000","last_active":"2023-07-14T12:01:06.807217+0000","last_peered":"2023-07-14T12:01:06.807217+0000","last_clean":"2023-07-14T12:01:06.807217+0000","last_became_active":"2023-07-14T12:00:06.812112+0000","last_became_peered":"2023-07-14T12:00:06.812112+0000","last_unstale":"2023-07-14T12:01:06.807217+0000","last_undegraded":"2023-07-14T12:01:06.807217+0000","last_fullsized":"2023-07-14T12:01:06.807217+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:36:45.024957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805908+0000","last_change":"2023-07-14T11:59:43.466351+0000","last_active":"2023-07-14T12:00:06.805908+0000","last_peered":"2023-07-14T12:00:06.805908+0000","last_clean":"2023-07-14T12:00:06.805908+0000","last_became_active":"2023-07-14T11:59:43.465442+0000","last_became_peered":"2023-07-14T11:59:43.465442+0000","last_unstale":"2023-07-14T12:00:06.805908+0000","last_undegraded":"2023-07-14T12:00:06.805908+0000","last_fullsized":"2023-07-14T12:00:06.805908+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:52:04.297945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.809112+0000","last_change":"2023-07-14T11:59:43.457684+0000","last_active":"2023-07-14T12:00:06.809112+0000","last_peered":"2023-07-14T12:00:06.809112+0000","last_clean":"2023-07-14T12:00:06.809112+0000","last_became_active":"2023-07-14T11:59:43.457442+0000","last_became_peered":"2023-07-14T11:59:43.457442+0000","last_unstale":"2023-07-14T12:00:06.809112+0000","last_undegraded":"2023-07-14T12:00:06.809112+0000","last_fullsized":"2023-07-14T12:00:06.809112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:06:41.592243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.328566+0000","last_change":"2023-07-14T11:59:43.463652+0000","last_active":"2023-07-14T12:00:07.328566+0000","last_peered":"2023-07-14T12:00:07.328566+0000","last_clean":"2023-07-14T12:00:07.328566+0000","last_became_active":"2023-07-14T11:59:43.463311+0000","last_became_peered":"2023-07-14T11:59:43.463311+0000","last_unstale":"2023-07-14T12:00:07.328566+0000","last_undegraded":"2023-07-14T12:00:07.328566+0000","last_fullsized":"2023-07-14T12:00:07.328566+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:28:35.177490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808344+0000","last_change":"2023-07-14T11:59:43.464633+0000","last_active":"2023-07-14T12:00:06.808344+0000","last_peered":"2023-07-14T12:00:06.808344+0000","last_clean":"2023-07-14T12:00:06.808344+0000","last_became_active":"2023-07-14T11:59:43.463847+0000","last_became_peered":"2023-07-14T11:59:43.463847+0000","last_unstale":"2023-07-14T12:00:06.808344+0000","last_undegraded":"2023-07-14T12:00:06.808344+0000","last_fullsized":"2023-07-14T12:00:06.808344+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:08:28.236596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808316+0000","last_change":"2023-07-14T11:59:43.465650+0000","last_active":"2023-07-14T12:00:06.808316+0000","last_peered":"2023-07-14T12:00:06.808316+0000","last_clean":"2023-07-14T12:00:06.808316+0000","last_became_active":"2023-07-14T11:59:43.465276+0000","last_became_peered":"2023-07-14T11:59:43.465276+0000","last_unstale":"2023-07-14T12:00:06.808316+0000","last_undegraded":"2023-07-14T12:00:06.808316+0000","last_fullsized":"2023-07-14T12:00:06.808316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:00:25.094505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.812937+0000","last_change":"2023-07-14T11:59:43.466437+0000","last_active":"2023-07-14T12:00:06.812937+0000","last_peered":"2023-07-14T12:00:06.812937+0000","last_clean":"2023-07-14T12:00:06.812937+0000","last_became_active":"2023-07-14T11:59:43.465361+0000","last_became_peered":"2023-07-14T11:59:43.465361+0000","last_unstale":"2023-07-14T12:00:06.812937+0000","last_undegraded":"2023-07-14T12:00:06.812937+0000","last_fullsized":"2023-07-14T12:00:06.812937+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:31:26.002175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.814761+0000","last_change":"2023-07-14T11:59:43.469452+0000","last_active":"2023-07-14T12:00:06.814761+0000","last_peered":"2023-07-14T12:00:06.814761+0000","last_clean":"2023-07-14T12:00:06.814761+0000","last_became_active":"2023-07-14T11:59:43.469112+0000","last_became_peered":"2023-07-14T11:59:43.469112+0000","last_unstale":"2023-07-14T12:00:06.814761+0000","last_undegraded":"2023-07-14T12:00:06.814761+0000","last_fullsized":"2023-07-14T12:00:06.814761+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:25:31.596988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.804882+0000","last_change":"2023-07-14T11:59:43.466746+0000","last_active":"2023-07-14T12:00:06.804882+0000","last_peered":"2023-07-14T12:00:06.804882+0000","last_clean":"2023-07-14T12:00:06.804882+0000","last_became_active":"2023-07-14T11:59:43.466530+0000","last_became_peered":"2023-07-14T11:59:43.466530+0000","last_unstale":"2023-07-14T12:00:06.804882+0000","last_undegraded":"2023-07-14T12:00:06.804882+0000","last_fullsized":"2023-07-14T12:00:06.804882+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:47:15.633490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.812795+0000","last_change":"2023-07-14T11:59:43.468510+0000","last_active":"2023-07-14T12:00:06.812795+0000","last_peered":"2023-07-14T12:00:06.812795+0000","last_clean":"2023-07-14T12:00:06.812795+0000","last_became_active":"2023-07-14T11:59:43.467990+0000","last_became_peered":"2023-07-14T11:59:43.467990+0000","last_unstale":"2023-07-14T12:00:06.812795+0000","last_undegraded":"2023-07-14T12:00:06.812795+0000","last_fullsized":"2023-07-14T12:00:06.812795+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:35:21.496112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-14T12:01:06.807853+0000","last_change":"2023-07-14T11:59:43.465290+0000","last_active":"2023-07-14T12:01:06.807853+0000","last_peered":"2023-07-14T12:01:06.807853+0000","last_clean":"2023-07-14T12:01:06.807853+0000","last_became_active":"2023-07-14T11:59:43.464982+0000","last_became_peered":"2023-07-14T11:59:43.464982+0000","last_unstale":"2023-07-14T12:01:06.807853+0000","last_undegraded":"2023-07-14T12:01:06.807853+0000","last_fullsized":"2023-07-14T12:01:06.807853+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:58:05.008569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-14T12:01:06.815053+0000","last_change":"2023-07-14T11:59:43.465053+0000","last_active":"2023-07-14T12:01:06.815053+0000","last_peered":"2023-07-14T12:01:06.815053+0000","last_clean":"2023-07-14T12:01:06.815053+0000","last_became_active":"2023-07-14T11:59:43.464759+0000","last_became_peered":"2023-07-14T11:59:43.464759+0000","last_unstale":"2023-07-14T12:01:06.815053+0000","last_undegraded":"2023-07-14T12:01:06.815053+0000","last_fullsized":"2023-07-14T12:01:06.815053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:33:14.338468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-14T12:01:06.814150+0000","last_change":"2023-07-14T11:59:47.792164+0000","last_active":"2023-07-14T12:01:06.814150+0000","last_peered":"2023-07-14T12:01:06.814150+0000","last_clean":"2023-07-14T12:01:06.814150+0000","last_became_active":"2023-07-14T11:59:47.791749+0000","last_became_peered":"2023-07-14T11:59:47.791749+0000","last_unstale":"2023-07-14T12:01:06.814150+0000","last_undegraded":"2023-07-14T12:01:06.814150+0000","last_fullsized":"2023-07-14T12:01:06.814150+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:42:40.666299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'13","reported_seq":92,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-14T12:01:06.808114+0000","last_change":"2023-07-14T11:59:42.759256+0000","last_active":"2023-07-14T12:01:06.808114+0000","last_peered":"2023-07-14T12:01:06.808114+0000","last_clean":"2023-07-14T12:01:06.808114+0000","last_became_active":"2023-07-14T11:59:42.758938+0000","last_became_peered":"2023-07-14T11:59:42.758938+0000","last_unstale":"2023-07-14T12:01:06.808114+0000","last_undegraded":"2023-07-14T12:01:06.808114+0000","last_fullsized":"2023-07-14T12:01:06.808114+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:19:10.316619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":51,"num_read_kb":51,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.811011+0000","last_change":"2023-07-14T11:59:43.467503+0000","last_active":"2023-07-14T12:00:06.811011+0000","last_peered":"2023-07-14T12:00:06.811011+0000","last_clean":"2023-07-14T12:00:06.811011+0000","last_became_active":"2023-07-14T11:59:43.467130+0000","last_became_peered":"2023-07-14T11:59:43.467130+0000","last_unstale":"2023-07-14T12:00:06.811011+0000","last_undegraded":"2023-07-14T12:00:06.811011+0000","last_fullsized":"2023-07-14T12:00:06.811011+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:26:08.037607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.807340+0000","last_change":"2023-07-14T11:59:47.783514+0000","last_active":"2023-07-14T12:00:06.807340+0000","last_peered":"2023-07-14T12:00:06.807340+0000","last_clean":"2023-07-14T12:00:06.807340+0000","last_became_active":"2023-07-14T11:59:47.783115+0000","last_became_peered":"2023-07-14T11:59:47.783115+0000","last_unstale":"2023-07-14T12:00:06.807340+0000","last_undegraded":"2023-07-14T12:00:06.807340+0000","last_fullsized":"2023-07-14T12:00:06.807340+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:19:30.538776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.811193+0000","last_change":"2023-07-14T11:59:42.768975+0000","last_active":"2023-07-14T12:00:06.811193+0000","last_peered":"2023-07-14T12:00:06.811193+0000","last_clean":"2023-07-14T12:00:06.811193+0000","last_became_active":"2023-07-14T11:59:42.768603+0000","last_became_peered":"2023-07-14T11:59:42.768603+0000","last_unstale":"2023-07-14T12:00:06.811193+0000","last_undegraded":"2023-07-14T12:00:06.811193+0000","last_fullsized":"2023-07-14T12:00:06.811193+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:20:32.308882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.813232+0000","last_change":"2023-07-14T11:59:43.474252+0000","last_active":"2023-07-14T12:00:06.813232+0000","last_peered":"2023-07-14T12:00:06.813232+0000","last_clean":"2023-07-14T12:00:06.813232+0000","last_became_active":"2023-07-14T11:59:43.473955+0000","last_became_peered":"2023-07-14T11:59:43.473955+0000","last_unstale":"2023-07-14T12:00:06.813232+0000","last_undegraded":"2023-07-14T12:00:06.813232+0000","last_fullsized":"2023-07-14T12:00:06.813232+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:52:46.217372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.813893+0000","last_change":"2023-07-14T12:00:06.813893+0000","last_active":"2023-07-14T12:00:06.813893+0000","last_peered":"2023-07-14T12:00:06.813893+0000","last_clean":"2023-07-14T12:00:06.813893+0000","last_became_active":"2023-07-14T12:00:06.813568+0000","last_became_peered":"2023-07-14T12:00:06.813568+0000","last_unstale":"2023-07-14T12:00:06.813893+0000","last_undegraded":"2023-07-14T12:00:06.813893+0000","last_fullsized":"2023-07-14T12:00:06.813893+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:57:58.591410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:17.499054+0000","last_change":"2023-07-14T11:59:42.760791+0000","last_active":"2023-07-14T12:00:17.499054+0000","last_peered":"2023-07-14T12:00:17.499054+0000","last_clean":"2023-07-14T12:00:17.499054+0000","last_became_active":"2023-07-14T11:59:42.760422+0000","last_became_peered":"2023-07-14T11:59:42.760422+0000","last_unstale":"2023-07-14T12:00:17.499054+0000","last_undegraded":"2023-07-14T12:00:17.499054+0000","last_fullsized":"2023-07-14T12:00:17.499054+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:10:21.008350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805235+0000","last_change":"2023-07-14T11:59:43.455829+0000","last_active":"2023-07-14T12:00:06.805235+0000","last_peered":"2023-07-14T12:00:06.805235+0000","last_clean":"2023-07-14T12:00:06.805235+0000","last_became_active":"2023-07-14T11:59:43.455541+0000","last_became_peered":"2023-07-14T11:59:43.455541+0000","last_unstale":"2023-07-14T12:00:06.805235+0000","last_undegraded":"2023-07-14T12:00:06.805235+0000","last_fullsized":"2023-07-14T12:00:06.805235+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:42:43.607658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.807537+0000","last_change":"2023-07-14T11:59:47.783496+0000","last_active":"2023-07-14T12:00:06.807537+0000","last_peered":"2023-07-14T12:00:06.807537+0000","last_clean":"2023-07-14T12:00:06.807537+0000","last_became_active":"2023-07-14T11:59:47.783128+0000","last_became_peered":"2023-07-14T11:59:47.783128+0000","last_unstale":"2023-07-14T12:00:06.807537+0000","last_undegraded":"2023-07-14T12:00:06.807537+0000","last_fullsized":"2023-07-14T12:00:06.807537+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:55:13.905682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:16.495600+0000","last_change":"2023-07-14T11:59:42.757255+0000","last_active":"2023-07-14T12:00:16.495600+0000","last_peered":"2023-07-14T12:00:16.495600+0000","last_clean":"2023-07-14T12:00:16.495600+0000","last_became_active":"2023-07-14T11:59:42.756882+0000","last_became_peered":"2023-07-14T11:59:42.756882+0000","last_unstale":"2023-07-14T12:00:16.495600+0000","last_undegraded":"2023-07-14T12:00:16.495600+0000","last_fullsized":"2023-07-14T12:00:16.495600+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:22:50.954980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808779+0000","last_change":"2023-07-14T11:59:43.465501+0000","last_active":"2023-07-14T12:00:06.808779+0000","last_peered":"2023-07-14T12:00:06.808779+0000","last_clean":"2023-07-14T12:00:06.808779+0000","last_became_active":"2023-07-14T11:59:43.465218+0000","last_became_peered":"2023-07-14T11:59:43.465218+0000","last_unstale":"2023-07-14T12:00:06.808779+0000","last_undegraded":"2023-07-14T12:00:06.808779+0000","last_fullsized":"2023-07-14T12:00:06.808779+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:02:45.983300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-14T12:01:06.815247+0000","last_change":"2023-07-14T11:59:47.777816+0000","last_active":"2023-07-14T12:01:06.815247+0000","last_peered":"2023-07-14T12:01:06.815247+0000","last_clean":"2023-07-14T12:01:06.815247+0000","last_became_active":"2023-07-14T11:59:47.777373+0000","last_became_peered":"2023-07-14T11:59:47.777373+0000","last_unstale":"2023-07-14T12:01:06.815247+0000","last_undegraded":"2023-07-14T12:01:06.815247+0000","last_fullsized":"2023-07-14T12:01:06.815247+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:09:21.325342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":81,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:01:03.565984+0000","last_change":"2023-07-14T11:59:42.756362+0000","last_active":"2023-07-14T12:01:03.565984+0000","last_peered":"2023-07-14T12:01:03.565984+0000","last_clean":"2023-07-14T12:01:03.565984+0000","last_became_active":"2023-07-14T11:59:42.756074+0000","last_became_peered":"2023-07-14T11:59:42.756074+0000","last_unstale":"2023-07-14T12:01:03.565984+0000","last_undegraded":"2023-07-14T12:01:03.565984+0000","last_fullsized":"2023-07-14T12:01:03.565984+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:22:56.214797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.812341+0000","last_change":"2023-07-14T12:00:06.812341+0000","last_active":"2023-07-14T12:00:06.812341+0000","last_peered":"2023-07-14T12:00:06.812341+0000","last_clean":"2023-07-14T12:00:06.812341+0000","last_became_active":"2023-07-14T12:00:06.811891+0000","last_became_peered":"2023-07-14T12:00:06.811891+0000","last_unstale":"2023-07-14T12:00:06.812341+0000","last_undegraded":"2023-07-14T12:00:06.812341+0000","last_fullsized":"2023-07-14T12:00:06.812341+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:19:54.770200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-14T12:01:06.814516+0000","last_change":"2023-07-14T11:59:47.785323+0000","last_active":"2023-07-14T12:01:06.814516+0000","last_peered":"2023-07-14T12:01:06.814516+0000","last_clean":"2023-07-14T12:01:06.814516+0000","last_became_active":"2023-07-14T11:59:47.784892+0000","last_became_peered":"2023-07-14T11:59:47.784892+0000","last_unstale":"2023-07-14T12:01:06.814516+0000","last_undegraded":"2023-07-14T12:01:06.814516+0000","last_fullsized":"2023-07-14T12:01:06.814516+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:17:57.601972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:15.490306+0000","last_change":"2023-07-14T11:59:42.756364+0000","last_active":"2023-07-14T12:00:15.490306+0000","last_peered":"2023-07-14T12:00:15.490306+0000","last_clean":"2023-07-14T12:00:15.490306+0000","last_became_active":"2023-07-14T11:59:42.756080+0000","last_became_peered":"2023-07-14T11:59:42.756080+0000","last_unstale":"2023-07-14T12:00:15.490306+0000","last_undegraded":"2023-07-14T12:00:15.490306+0000","last_fullsized":"2023-07-14T12:00:15.490306+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:55:28.974638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808819+0000","last_change":"2023-07-14T11:59:43.470331+0000","last_active":"2023-07-14T12:00:06.808819+0000","last_peered":"2023-07-14T12:00:06.808819+0000","last_clean":"2023-07-14T12:00:06.808819+0000","last_became_active":"2023-07-14T11:59:43.469806+0000","last_became_peered":"2023-07-14T11:59:43.469806+0000","last_unstale":"2023-07-14T12:00:06.808819+0000","last_undegraded":"2023-07-14T12:00:06.808819+0000","last_fullsized":"2023-07-14T12:00:06.808819+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:31:39.648120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.812748+0000","last_change":"2023-07-14T11:59:47.784083+0000","last_active":"2023-07-14T12:00:06.812748+0000","last_peered":"2023-07-14T12:00:06.812748+0000","last_clean":"2023-07-14T12:00:06.812748+0000","last_became_active":"2023-07-14T11:59:47.783624+0000","last_became_peered":"2023-07-14T11:59:47.783624+0000","last_unstale":"2023-07-14T12:00:06.812748+0000","last_undegraded":"2023-07-14T12:00:06.812748+0000","last_fullsized":"2023-07-14T12:00:06.812748+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:19:59.493479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:14.477781+0000","last_change":"2023-07-14T11:59:42.764869+0000","last_active":"2023-07-14T12:00:14.477781+0000","last_peered":"2023-07-14T12:00:14.477781+0000","last_clean":"2023-07-14T12:00:14.477781+0000","last_became_active":"2023-07-14T11:59:42.764514+0000","last_became_peered":"2023-07-14T11:59:42.764514+0000","last_unstale":"2023-07-14T12:00:14.477781+0000","last_undegraded":"2023-07-14T12:00:14.477781+0000","last_fullsized":"2023-07-14T12:00:14.477781+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:14:29.041985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.809908+0000","last_change":"2023-07-14T11:59:43.458165+0000","last_active":"2023-07-14T12:00:06.809908+0000","last_peered":"2023-07-14T12:00:06.809908+0000","last_clean":"2023-07-14T12:00:06.809908+0000","last_became_active":"2023-07-14T11:59:43.457829+0000","last_became_peered":"2023-07-14T11:59:43.457829+0000","last_unstale":"2023-07-14T12:00:06.809908+0000","last_undegraded":"2023-07-14T12:00:06.809908+0000","last_fullsized":"2023-07-14T12:00:06.809908+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:43:38.283487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.814633+0000","last_change":"2023-07-14T11:59:47.782676+0000","last_active":"2023-07-14T12:00:06.814633+0000","last_peered":"2023-07-14T12:00:06.814633+0000","last_clean":"2023-07-14T12:00:06.814633+0000","last_became_active":"2023-07-14T11:59:47.782283+0000","last_became_peered":"2023-07-14T11:59:47.782283+0000","last_unstale":"2023-07-14T12:00:06.814633+0000","last_undegraded":"2023-07-14T12:00:06.814633+0000","last_fullsized":"2023-07-14T12:00:06.814633+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:50:29.299244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:16.494916+0000","last_change":"2023-07-14T11:59:42.756701+0000","last_active":"2023-07-14T12:00:16.494916+0000","last_peered":"2023-07-14T12:00:16.494916+0000","last_clean":"2023-07-14T12:00:16.494916+0000","last_became_active":"2023-07-14T11:59:42.756452+0000","last_became_peered":"2023-07-14T11:59:42.756452+0000","last_unstale":"2023-07-14T12:00:16.494916+0000","last_undegraded":"2023-07-14T12:00:16.494916+0000","last_fullsized":"2023-07-14T12:00:16.494916+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:56:37.979449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-14T12:01:06.807397+0000","last_change":"2023-07-14T11:59:43.457029+0000","last_active":"2023-07-14T12:01:06.807397+0000","last_peered":"2023-07-14T12:01:06.807397+0000","last_clean":"2023-07-14T12:01:06.807397+0000","last_became_active":"2023-07-14T11:59:43.456710+0000","last_became_peered":"2023-07-14T11:59:43.456710+0000","last_unstale":"2023-07-14T12:01:06.807397+0000","last_undegraded":"2023-07-14T12:01:06.807397+0000","last_fullsized":"2023-07-14T12:01:06.807397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:25:08.050556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.811649+0000","last_change":"2023-07-14T11:59:47.785732+0000","last_active":"2023-07-14T12:00:06.811649+0000","last_peered":"2023-07-14T12:00:06.811649+0000","last_clean":"2023-07-14T12:00:06.811649+0000","last_became_active":"2023-07-14T11:59:47.785289+0000","last_became_peered":"2023-07-14T11:59:47.785289+0000","last_unstale":"2023-07-14T12:00:06.811649+0000","last_undegraded":"2023-07-14T12:00:06.811649+0000","last_fullsized":"2023-07-14T12:00:06.811649+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:58:47.873312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":124,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:01:03.565696+0000","last_change":"2023-07-14T11:59:42.760013+0000","last_active":"2023-07-14T12:01:03.565696+0000","last_peered":"2023-07-14T12:01:03.565696+0000","last_clean":"2023-07-14T12:01:03.565696+0000","last_became_active":"2023-07-14T11:59:42.759673+0000","last_became_peered":"2023-07-14T11:59:42.759673+0000","last_unstale":"2023-07-14T12:01:03.565696+0000","last_undegraded":"2023-07-14T12:01:03.565696+0000","last_fullsized":"2023-07-14T12:01:03.565696+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:31:23.656642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":94,"num_read_kb":94,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805922+0000","last_change":"2023-07-14T11:59:43.455418+0000","last_active":"2023-07-14T12:00:06.805922+0000","last_peered":"2023-07-14T12:00:06.805922+0000","last_clean":"2023-07-14T12:00:06.805922+0000","last_became_active":"2023-07-14T11:59:43.454963+0000","last_became_peered":"2023-07-14T11:59:43.454963+0000","last_unstale":"2023-07-14T12:00:06.805922+0000","last_undegraded":"2023-07-14T12:00:06.805922+0000","last_fullsized":"2023-07-14T12:00:06.805922+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:35:27.262942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-14T12:01:06.807364+0000","last_change":"2023-07-14T11:59:47.784330+0000","last_active":"2023-07-14T12:01:06.807364+0000","last_peered":"2023-07-14T12:01:06.807364+0000","last_clean":"2023-07-14T12:01:06.807364+0000","last_became_active":"2023-07-14T11:59:47.783640+0000","last_became_peered":"2023-07-14T11:59:47.783640+0000","last_unstale":"2023-07-14T12:01:06.807364+0000","last_undegraded":"2023-07-14T12:01:06.807364+0000","last_fullsized":"2023-07-14T12:01:06.807364+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:58:53.481976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:16.496542+0000","last_change":"2023-07-14T11:59:42.758856+0000","last_active":"2023-07-14T12:00:16.496542+0000","last_peered":"2023-07-14T12:00:16.496542+0000","last_clean":"2023-07-14T12:00:16.496542+0000","last_became_active":"2023-07-14T11:59:42.758571+0000","last_became_peered":"2023-07-14T11:59:42.758571+0000","last_unstale":"2023-07-14T12:00:16.496542+0000","last_undegraded":"2023-07-14T12:00:16.496542+0000","last_fullsized":"2023-07-14T12:00:16.496542+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:22:26.607862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.814254+0000","last_change":"2023-07-14T11:59:43.466302+0000","last_active":"2023-07-14T12:00:06.814254+0000","last_peered":"2023-07-14T12:00:06.814254+0000","last_clean":"2023-07-14T12:00:06.814254+0000","last_became_active":"2023-07-14T11:59:43.465958+0000","last_became_peered":"2023-07-14T11:59:43.465958+0000","last_unstale":"2023-07-14T12:00:06.814254+0000","last_undegraded":"2023-07-14T12:00:06.814254+0000","last_fullsized":"2023-07-14T12:00:06.814254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:09:00.018902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.813066+0000","last_change":"2023-07-14T12:00:06.813066+0000","last_active":"2023-07-14T12:00:06.813066+0000","last_peered":"2023-07-14T12:00:06.813066+0000","last_clean":"2023-07-14T12:00:06.813066+0000","last_became_active":"2023-07-14T12:00:06.812460+0000","last_became_peered":"2023-07-14T12:00:06.812460+0000","last_unstale":"2023-07-14T12:00:06.813066+0000","last_undegraded":"2023-07-14T12:00:06.813066+0000","last_fullsized":"2023-07-14T12:00:06.813066+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:47:40.893162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:15.487525+0000","last_change":"2023-07-14T11:59:42.765777+0000","last_active":"2023-07-14T12:00:15.487525+0000","last_peered":"2023-07-14T12:00:15.487525+0000","last_clean":"2023-07-14T12:00:15.487525+0000","last_became_active":"2023-07-14T11:59:42.765536+0000","last_became_peered":"2023-07-14T11:59:42.765536+0000","last_unstale":"2023-07-14T12:00:15.487525+0000","last_undegraded":"2023-07-14T12:00:15.487525+0000","last_fullsized":"2023-07-14T12:00:15.487525+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:27:05.053926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805426+0000","last_change":"2023-07-14T11:59:43.463665+0000","last_active":"2023-07-14T12:00:06.805426+0000","last_peered":"2023-07-14T12:00:06.805426+0000","last_clean":"2023-07-14T12:00:06.805426+0000","last_became_active":"2023-07-14T11:59:43.463368+0000","last_became_peered":"2023-07-14T11:59:43.463368+0000","last_unstale":"2023-07-14T12:00:06.805426+0000","last_undegraded":"2023-07-14T12:00:06.805426+0000","last_fullsized":"2023-07-14T12:00:06.805426+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:49:21.359682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-14T12:01:06.807579+0000","last_change":"2023-07-14T11:59:47.780706+0000","last_active":"2023-07-14T12:01:06.807579+0000","last_peered":"2023-07-14T12:01:06.807579+0000","last_clean":"2023-07-14T12:01:06.807579+0000","last_became_active":"2023-07-14T11:59:47.780207+0000","last_became_peered":"2023-07-14T11:59:47.780207+0000","last_unstale":"2023-07-14T12:01:06.807579+0000","last_undegraded":"2023-07-14T12:01:06.807579+0000","last_fullsized":"2023-07-14T12:01:06.807579+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:03:05.372695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'14","reported_seq":136,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:01:03.565441+0000","last_change":"2023-07-14T11:59:42.757238+0000","last_active":"2023-07-14T12:01:03.565441+0000","last_peered":"2023-07-14T12:01:03.565441+0000","last_clean":"2023-07-14T12:01:03.565441+0000","last_became_active":"2023-07-14T11:59:42.756887+0000","last_became_peered":"2023-07-14T11:59:42.756887+0000","last_unstale":"2023-07-14T12:01:03.565441+0000","last_undegraded":"2023-07-14T12:01:03.565441+0000","last_fullsized":"2023-07-14T12:01:03.565441+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:09:27.857990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3794,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":96,"num_read_kb":52,"num_write":16,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808529+0000","last_change":"2023-07-14T11:59:43.460741+0000","last_active":"2023-07-14T12:00:06.808529+0000","last_peered":"2023-07-14T12:00:06.808529+0000","last_clean":"2023-07-14T12:00:06.808529+0000","last_became_active":"2023-07-14T11:59:43.460423+0000","last_became_peered":"2023-07-14T11:59:43.460423+0000","last_unstale":"2023-07-14T12:00:06.808529+0000","last_undegraded":"2023-07-14T12:00:06.808529+0000","last_fullsized":"2023-07-14T12:00:06.808529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:26:56.645695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.809254+0000","last_change":"2023-07-14T11:59:47.781321+0000","last_active":"2023-07-14T12:00:06.809254+0000","last_peered":"2023-07-14T12:00:06.809254+0000","last_clean":"2023-07-14T12:00:06.809254+0000","last_became_active":"2023-07-14T11:59:47.780941+0000","last_became_peered":"2023-07-14T11:59:47.780941+0000","last_unstale":"2023-07-14T12:00:06.809254+0000","last_undegraded":"2023-07-14T12:00:06.809254+0000","last_fullsized":"2023-07-14T12:00:06.809254+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:25:26.671430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-14T12:01:06.814400+0000","last_change":"2023-07-14T11:59:42.759403+0000","last_active":"2023-07-14T12:01:06.814400+0000","last_peered":"2023-07-14T12:01:06.814400+0000","last_clean":"2023-07-14T12:01:06.814400+0000","last_became_active":"2023-07-14T11:59:42.759036+0000","last_became_peered":"2023-07-14T11:59:42.759036+0000","last_unstale":"2023-07-14T12:01:06.814400+0000","last_undegraded":"2023-07-14T12:01:06.814400+0000","last_fullsized":"2023-07-14T12:01:06.814400+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:26:28.023021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808124+0000","last_change":"2023-07-14T11:59:43.457644+0000","last_active":"2023-07-14T12:00:06.808124+0000","last_peered":"2023-07-14T12:00:06.808124+0000","last_clean":"2023-07-14T12:00:06.808124+0000","last_became_active":"2023-07-14T11:59:43.457363+0000","last_became_peered":"2023-07-14T11:59:43.457363+0000","last_unstale":"2023-07-14T12:00:06.808124+0000","last_undegraded":"2023-07-14T12:00:06.808124+0000","last_fullsized":"2023-07-14T12:00:06.808124+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:12:17.222392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.812650+0000","last_change":"2023-07-14T11:59:47.789684+0000","last_active":"2023-07-14T12:00:06.812650+0000","last_peered":"2023-07-14T12:00:06.812650+0000","last_clean":"2023-07-14T12:00:06.812650+0000","last_became_active":"2023-07-14T11:59:47.789226+0000","last_became_peered":"2023-07-14T11:59:47.789226+0000","last_unstale":"2023-07-14T12:00:06.812650+0000","last_undegraded":"2023-07-14T12:00:06.812650+0000","last_fullsized":"2023-07-14T12:00:06.812650+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:56:58.961737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:16.494919+0000","last_change":"2023-07-14T11:59:42.751497+0000","last_active":"2023-07-14T12:00:16.494919+0000","last_peered":"2023-07-14T12:00:16.494919+0000","last_clean":"2023-07-14T12:00:16.494919+0000","last_became_active":"2023-07-14T11:59:42.751217+0000","last_became_peered":"2023-07-14T11:59:42.751217+0000","last_unstale":"2023-07-14T12:00:16.494919+0000","last_undegraded":"2023-07-14T12:00:16.494919+0000","last_fullsized":"2023-07-14T12:00:16.494919+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:53:00.140364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.328171+0000","last_change":"2023-07-14T11:59:43.462359+0000","last_active":"2023-07-14T12:00:07.328171+0000","last_peered":"2023-07-14T12:00:07.328171+0000","last_clean":"2023-07-14T12:00:07.328171+0000","last_became_active":"2023-07-14T11:59:43.461781+0000","last_became_peered":"2023-07-14T11:59:43.461781+0000","last_unstale":"2023-07-14T12:00:07.328171+0000","last_undegraded":"2023-07-14T12:00:07.328171+0000","last_fullsized":"2023-07-14T12:00:07.328171+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:10:22.527860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-14T12:01:06.807869+0000","last_change":"2023-07-14T11:59:47.790173+0000","last_active":"2023-07-14T12:01:06.807869+0000","last_peered":"2023-07-14T12:01:06.807869+0000","last_clean":"2023-07-14T12:01:06.807869+0000","last_became_active":"2023-07-14T11:59:47.789418+0000","last_became_peered":"2023-07-14T11:59:47.789418+0000","last_unstale":"2023-07-14T12:01:06.807869+0000","last_undegraded":"2023-07-14T12:01:06.807869+0000","last_fullsized":"2023-07-14T12:01:06.807869+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:09:04.373583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'5","reported_seq":81,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:01:03.566014+0000","last_change":"2023-07-14T11:59:42.753068+0000","last_active":"2023-07-14T12:01:03.566014+0000","last_peered":"2023-07-14T12:01:03.566014+0000","last_clean":"2023-07-14T12:01:03.566014+0000","last_became_active":"2023-07-14T11:59:42.752794+0000","last_became_peered":"2023-07-14T11:59:42.752794+0000","last_unstale":"2023-07-14T12:01:03.566014+0000","last_undegraded":"2023-07-14T12:01:03.566014+0000","last_fullsized":"2023-07-14T12:01:03.566014+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:31:01.258740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":51,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.328012+0000","last_change":"2023-07-14T11:59:43.468655+0000","last_active":"2023-07-14T12:00:07.328012+0000","last_peered":"2023-07-14T12:00:07.328012+0000","last_clean":"2023-07-14T12:00:07.328012+0000","last_became_active":"2023-07-14T11:59:43.468361+0000","last_became_peered":"2023-07-14T11:59:43.468361+0000","last_unstale":"2023-07-14T12:00:07.328012+0000","last_undegraded":"2023-07-14T12:00:07.328012+0000","last_fullsized":"2023-07-14T12:00:07.328012+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:02:47.181093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808379+0000","last_change":"2023-07-14T11:59:47.783644+0000","last_active":"2023-07-14T12:00:06.808379+0000","last_peered":"2023-07-14T12:00:06.808379+0000","last_clean":"2023-07-14T12:00:06.808379+0000","last_became_active":"2023-07-14T11:59:47.783229+0000","last_became_peered":"2023-07-14T11:59:47.783229+0000","last_unstale":"2023-07-14T12:00:06.808379+0000","last_undegraded":"2023-07-14T12:00:06.808379+0000","last_fullsized":"2023-07-14T12:00:06.808379+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:15:35.788258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:16.494745+0000","last_change":"2023-07-14T11:59:42.768482+0000","last_active":"2023-07-14T12:00:16.494745+0000","last_peered":"2023-07-14T12:00:16.494745+0000","last_clean":"2023-07-14T12:00:16.494745+0000","last_became_active":"2023-07-14T11:59:42.768201+0000","last_became_peered":"2023-07-14T11:59:42.768201+0000","last_unstale":"2023-07-14T12:00:16.494745+0000","last_undegraded":"2023-07-14T12:00:16.494745+0000","last_fullsized":"2023-07-14T12:00:16.494745+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:39:29.438053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808385+0000","last_change":"2023-07-14T11:59:43.469651+0000","last_active":"2023-07-14T12:00:06.808385+0000","last_peered":"2023-07-14T12:00:06.808385+0000","last_clean":"2023-07-14T12:00:06.808385+0000","last_became_active":"2023-07-14T11:59:43.465848+0000","last_became_peered":"2023-07-14T11:59:43.465848+0000","last_unstale":"2023-07-14T12:00:06.808385+0000","last_undegraded":"2023-07-14T12:00:06.808385+0000","last_fullsized":"2023-07-14T12:00:06.808385+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:17:18.313257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.817649+0000","last_change":"2023-07-14T12:00:06.817110+0000","last_active":"2023-07-14T12:00:06.817649+0000","last_peered":"2023-07-14T12:00:06.817649+0000","last_clean":"2023-07-14T12:00:06.817649+0000","last_became_active":"2023-07-14T12:00:06.816587+0000","last_became_peered":"2023-07-14T12:00:06.816587+0000","last_unstale":"2023-07-14T12:00:06.817649+0000","last_undegraded":"2023-07-14T12:00:06.817649+0000","last_fullsized":"2023-07-14T12:00:06.817649+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:04:52.873643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,0,3,1],"acting":[10,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:17.497708+0000","last_change":"2023-07-14T11:59:42.758106+0000","last_active":"2023-07-14T12:00:17.497708+0000","last_peered":"2023-07-14T12:00:17.497708+0000","last_clean":"2023-07-14T12:00:17.497708+0000","last_became_active":"2023-07-14T11:59:42.757840+0000","last_became_peered":"2023-07-14T11:59:42.757840+0000","last_unstale":"2023-07-14T12:00:17.497708+0000","last_undegraded":"2023-07-14T12:00:17.497708+0000","last_fullsized":"2023-07-14T12:00:17.497708+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:12:46.164720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.812652+0000","last_change":"2023-07-14T11:59:43.467800+0000","last_active":"2023-07-14T12:00:06.812652+0000","last_peered":"2023-07-14T12:00:06.812652+0000","last_clean":"2023-07-14T12:00:06.812652+0000","last_became_active":"2023-07-14T11:59:43.467448+0000","last_became_peered":"2023-07-14T11:59:43.467448+0000","last_unstale":"2023-07-14T12:00:06.812652+0000","last_undegraded":"2023-07-14T12:00:06.812652+0000","last_fullsized":"2023-07-14T12:00:06.812652+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:34:25.781268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.807574+0000","last_change":"2023-07-14T11:59:47.915884+0000","last_active":"2023-07-14T12:00:06.807574+0000","last_peered":"2023-07-14T12:00:06.807574+0000","last_clean":"2023-07-14T12:00:06.807574+0000","last_became_active":"2023-07-14T11:59:47.915449+0000","last_became_peered":"2023-07-14T11:59:47.915449+0000","last_unstale":"2023-07-14T12:00:06.807574+0000","last_undegraded":"2023-07-14T12:00:06.807574+0000","last_fullsized":"2023-07-14T12:00:06.807574+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:41:55.813157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-14T12:01:06.807622+0000","last_change":"2023-07-14T11:59:42.771581+0000","last_active":"2023-07-14T12:01:06.807622+0000","last_peered":"2023-07-14T12:01:06.807622+0000","last_clean":"2023-07-14T12:01:06.807622+0000","last_became_active":"2023-07-14T11:59:42.771228+0000","last_became_peered":"2023-07-14T11:59:42.771228+0000","last_unstale":"2023-07-14T12:01:06.807622+0000","last_undegraded":"2023-07-14T12:01:06.807622+0000","last_fullsized":"2023-07-14T12:01:06.807622+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:27:15.764768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808751+0000","last_change":"2023-07-14T11:59:43.460987+0000","last_active":"2023-07-14T12:00:06.808751+0000","last_peered":"2023-07-14T12:00:06.808751+0000","last_clean":"2023-07-14T12:00:06.808751+0000","last_became_active":"2023-07-14T11:59:43.460604+0000","last_became_peered":"2023-07-14T11:59:43.460604+0000","last_unstale":"2023-07-14T12:00:06.808751+0000","last_undegraded":"2023-07-14T12:00:06.808751+0000","last_fullsized":"2023-07-14T12:00:06.808751+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:00:15.767598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808575+0000","last_change":"2023-07-14T11:59:47.788604+0000","last_active":"2023-07-14T12:00:06.808575+0000","last_peered":"2023-07-14T12:00:06.808575+0000","last_clean":"2023-07-14T12:00:06.808575+0000","last_became_active":"2023-07-14T11:59:47.787854+0000","last_became_peered":"2023-07-14T11:59:47.787854+0000","last_unstale":"2023-07-14T12:00:06.808575+0000","last_undegraded":"2023-07-14T12:00:06.808575+0000","last_fullsized":"2023-07-14T12:00:06.808575+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:19:18.510764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'5","reported_seq":79,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:01:02.564666+0000","last_change":"2023-07-14T11:59:42.768985+0000","last_active":"2023-07-14T12:01:02.564666+0000","last_peered":"2023-07-14T12:01:02.564666+0000","last_clean":"2023-07-14T12:01:02.564666+0000","last_became_active":"2023-07-14T11:59:42.768688+0000","last_became_peered":"2023-07-14T11:59:42.768688+0000","last_unstale":"2023-07-14T12:01:02.564666+0000","last_undegraded":"2023-07-14T12:01:02.564666+0000","last_fullsized":"2023-07-14T12:01:02.564666+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:20:55.661084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":48,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.807718+0000","last_change":"2023-07-14T11:59:43.465529+0000","last_active":"2023-07-14T12:00:06.807718+0000","last_peered":"2023-07-14T12:00:06.807718+0000","last_clean":"2023-07-14T12:00:06.807718+0000","last_became_active":"2023-07-14T11:59:43.465206+0000","last_became_peered":"2023-07-14T11:59:43.465206+0000","last_unstale":"2023-07-14T12:00:06.807718+0000","last_undegraded":"2023-07-14T12:00:06.807718+0000","last_fullsized":"2023-07-14T12:00:06.807718+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:52:33.760607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808309+0000","last_change":"2023-07-14T11:59:47.915842+0000","last_active":"2023-07-14T12:00:06.808309+0000","last_peered":"2023-07-14T12:00:06.808309+0000","last_clean":"2023-07-14T12:00:06.808309+0000","last_became_active":"2023-07-14T11:59:47.915304+0000","last_became_peered":"2023-07-14T11:59:47.915304+0000","last_unstale":"2023-07-14T12:00:06.808309+0000","last_undegraded":"2023-07-14T12:00:06.808309+0000","last_fullsized":"2023-07-14T12:00:06.808309+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:26:43.397241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.809764+0000","last_change":"2023-07-14T11:59:43.472296+0000","last_active":"2023-07-14T12:00:06.809764+0000","last_peered":"2023-07-14T12:00:06.809764+0000","last_clean":"2023-07-14T12:00:06.809764+0000","last_became_active":"2023-07-14T11:59:43.470569+0000","last_became_peered":"2023-07-14T11:59:43.470569+0000","last_unstale":"2023-07-14T12:00:06.809764+0000","last_undegraded":"2023-07-14T12:00:06.809764+0000","last_fullsized":"2023-07-14T12:00:06.809764+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:06:23.546910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":82,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:01:05.567878+0000","last_change":"2023-07-14T11:59:42.769225+0000","last_active":"2023-07-14T12:01:05.567878+0000","last_peered":"2023-07-14T12:01:05.567878+0000","last_clean":"2023-07-14T12:01:05.567878+0000","last_became_active":"2023-07-14T11:59:42.768877+0000","last_became_peered":"2023-07-14T11:59:42.768877+0000","last_unstale":"2023-07-14T12:01:05.567878+0000","last_undegraded":"2023-07-14T12:01:05.567878+0000","last_fullsized":"2023-07-14T12:01:05.567878+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:19:23.782648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":51,"num_read_kb":51,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.328375+0000","last_change":"2023-07-14T11:59:47.792665+0000","last_active":"2023-07-14T12:00:07.328375+0000","last_peered":"2023-07-14T12:00:07.328375+0000","last_clean":"2023-07-14T12:00:07.328375+0000","last_became_active":"2023-07-14T11:59:47.792308+0000","last_became_peered":"2023-07-14T11:59:47.792308+0000","last_unstale":"2023-07-14T12:00:07.328375+0000","last_undegraded":"2023-07-14T12:00:07.328375+0000","last_fullsized":"2023-07-14T12:00:07.328375+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:18:05.628049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.810583+0000","last_change":"2023-07-14T11:59:43.461103+0000","last_active":"2023-07-14T12:00:06.810583+0000","last_peered":"2023-07-14T12:00:06.810583+0000","last_clean":"2023-07-14T12:00:06.810583+0000","last_became_active":"2023-07-14T11:59:43.460756+0000","last_became_peered":"2023-07-14T11:59:43.460756+0000","last_unstale":"2023-07-14T12:00:06.810583+0000","last_undegraded":"2023-07-14T12:00:06.810583+0000","last_fullsized":"2023-07-14T12:00:06.810583+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:05:41.752001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:17.497582+0000","last_change":"2023-07-14T11:59:42.752680+0000","last_active":"2023-07-14T12:00:17.497582+0000","last_peered":"2023-07-14T12:00:17.497582+0000","last_clean":"2023-07-14T12:00:17.497582+0000","last_became_active":"2023-07-14T11:59:42.752305+0000","last_became_peered":"2023-07-14T11:59:42.752305+0000","last_unstale":"2023-07-14T12:00:17.497582+0000","last_undegraded":"2023-07-14T12:00:17.497582+0000","last_fullsized":"2023-07-14T12:00:17.497582+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:01:57.359059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":353,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.327820+0000","last_change":"2023-07-14T11:58:37.124871+0000","last_active":"2023-07-14T12:00:07.327820+0000","last_peered":"2023-07-14T12:00:07.327820+0000","last_clean":"2023-07-14T12:00:07.327820+0000","last_became_active":"2023-07-14T11:58:37.124398+0000","last_became_peered":"2023-07-14T11:58:37.124398+0000","last_unstale":"2023-07-14T12:00:07.327820+0000","last_undegraded":"2023-07-14T12:00:07.327820+0000","last_fullsized":"2023-07-14T12:00:07.327820+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:55:28.287160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:55:28.287160+0000","last_clean_scrub_stamp":"2023-07-14T11:55:28.287160+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:00:38.717405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.815687+0000","last_change":"2023-07-14T12:00:06.815687+0000","last_active":"2023-07-14T12:00:06.815687+0000","last_peered":"2023-07-14T12:00:06.815687+0000","last_clean":"2023-07-14T12:00:06.815687+0000","last_became_active":"2023-07-14T12:00:06.815281+0000","last_became_peered":"2023-07-14T12:00:06.815281+0000","last_unstale":"2023-07-14T12:00:06.815687+0000","last_undegraded":"2023-07-14T12:00:06.815687+0000","last_fullsized":"2023-07-14T12:00:06.815687+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:04:42.469562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.813952+0000","last_change":"2023-07-14T12:00:06.813952+0000","last_active":"2023-07-14T12:00:06.813952+0000","last_peered":"2023-07-14T12:00:06.813952+0000","last_clean":"2023-07-14T12:00:06.813952+0000","last_became_active":"2023-07-14T12:00:06.813581+0000","last_became_peered":"2023-07-14T12:00:06.813581+0000","last_unstale":"2023-07-14T12:00:06.813952+0000","last_undegraded":"2023-07-14T12:00:06.813952+0000","last_fullsized":"2023-07-14T12:00:06.813952+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:12:06.783097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:14.479090+0000","last_change":"2023-07-14T11:59:42.757400+0000","last_active":"2023-07-14T12:00:14.479090+0000","last_peered":"2023-07-14T12:00:14.479090+0000","last_clean":"2023-07-14T12:00:14.479090+0000","last_became_active":"2023-07-14T11:59:42.757047+0000","last_became_peered":"2023-07-14T11:59:42.757047+0000","last_unstale":"2023-07-14T12:00:14.479090+0000","last_undegraded":"2023-07-14T12:00:14.479090+0000","last_fullsized":"2023-07-14T12:00:14.479090+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:42:39.292073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.811396+0000","last_change":"2023-07-14T11:59:47.788486+0000","last_active":"2023-07-14T12:00:06.811396+0000","last_peered":"2023-07-14T12:00:06.811396+0000","last_clean":"2023-07-14T12:00:06.811396+0000","last_became_active":"2023-07-14T11:59:47.787995+0000","last_became_peered":"2023-07-14T11:59:47.787995+0000","last_unstale":"2023-07-14T12:00:06.811396+0000","last_undegraded":"2023-07-14T12:00:06.811396+0000","last_fullsized":"2023-07-14T12:00:06.811396+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:01:52.099573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:16.495497+0000","last_change":"2023-07-14T11:59:42.765810+0000","last_active":"2023-07-14T12:00:16.495497+0000","last_peered":"2023-07-14T12:00:16.495497+0000","last_clean":"2023-07-14T12:00:16.495497+0000","last_became_active":"2023-07-14T11:59:42.765528+0000","last_became_peered":"2023-07-14T11:59:42.765528+0000","last_unstale":"2023-07-14T12:00:16.495497+0000","last_undegraded":"2023-07-14T12:00:16.495497+0000","last_fullsized":"2023-07-14T12:00:16.495497+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:18:02.321077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808724+0000","last_change":"2023-07-14T11:59:43.463763+0000","last_active":"2023-07-14T12:00:06.808724+0000","last_peered":"2023-07-14T12:00:06.808724+0000","last_clean":"2023-07-14T12:00:06.808724+0000","last_became_active":"2023-07-14T11:59:43.463411+0000","last_became_peered":"2023-07-14T11:59:43.463411+0000","last_unstale":"2023-07-14T12:00:06.808724+0000","last_undegraded":"2023-07-14T12:00:06.808724+0000","last_fullsized":"2023-07-14T12:00:06.808724+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:57:24.209856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.809720+0000","last_change":"2023-07-14T11:59:47.916405+0000","last_active":"2023-07-14T12:00:06.809720+0000","last_peered":"2023-07-14T12:00:06.809720+0000","last_clean":"2023-07-14T12:00:06.809720+0000","last_became_active":"2023-07-14T11:59:47.915892+0000","last_became_peered":"2023-07-14T11:59:47.915892+0000","last_unstale":"2023-07-14T12:00:06.809720+0000","last_undegraded":"2023-07-14T12:00:06.809720+0000","last_fullsized":"2023-07-14T12:00:06.809720+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:58:28.977188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'7","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-14T12:01:06.807937+0000","last_change":"2023-07-14T11:59:42.754848+0000","last_active":"2023-07-14T12:01:06.807937+0000","last_peered":"2023-07-14T12:01:06.807937+0000","last_clean":"2023-07-14T12:01:06.807937+0000","last_became_active":"2023-07-14T11:59:42.754522+0000","last_became_peered":"2023-07-14T11:59:42.754522+0000","last_unstale":"2023-07-14T12:01:06.807937+0000","last_undegraded":"2023-07-14T12:01:06.807937+0000","last_fullsized":"2023-07-14T12:01:06.807937+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:40:21.013054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.815209+0000","last_change":"2023-07-14T11:59:43.458954+0000","last_active":"2023-07-14T12:00:06.815209+0000","last_peered":"2023-07-14T12:00:06.815209+0000","last_clean":"2023-07-14T12:00:06.815209+0000","last_became_active":"2023-07-14T11:59:43.458631+0000","last_became_peered":"2023-07-14T11:59:43.458631+0000","last_unstale":"2023-07-14T12:00:06.815209+0000","last_undegraded":"2023-07-14T12:00:06.815209+0000","last_fullsized":"2023-07-14T12:00:06.815209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:41:19.853895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-14T12:01:06.814021+0000","last_change":"2023-07-14T11:59:47.776879+0000","last_active":"2023-07-14T12:01:06.814021+0000","last_peered":"2023-07-14T12:01:06.814021+0000","last_clean":"2023-07-14T12:01:06.814021+0000","last_became_active":"2023-07-14T11:59:47.776514+0000","last_became_peered":"2023-07-14T11:59:47.776514+0000","last_unstale":"2023-07-14T12:01:06.814021+0000","last_undegraded":"2023-07-14T12:01:06.814021+0000","last_fullsized":"2023-07-14T12:01:06.814021+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:55:27.606109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:16.496298+0000","last_change":"2023-07-14T11:59:42.754657+0000","last_active":"2023-07-14T12:00:16.496298+0000","last_peered":"2023-07-14T12:00:16.496298+0000","last_clean":"2023-07-14T12:00:16.496298+0000","last_became_active":"2023-07-14T11:59:42.754355+0000","last_became_peered":"2023-07-14T11:59:42.754355+0000","last_unstale":"2023-07-14T12:00:16.496298+0000","last_undegraded":"2023-07-14T12:00:16.496298+0000","last_fullsized":"2023-07-14T12:00:16.496298+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:16:59.044903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.329782+0000","last_change":"2023-07-14T12:00:07.329782+0000","last_active":"2023-07-14T12:00:07.329782+0000","last_peered":"2023-07-14T12:00:07.329782+0000","last_clean":"2023-07-14T12:00:07.329782+0000","last_became_active":"2023-07-14T12:00:07.329274+0000","last_became_peered":"2023-07-14T12:00:07.329274+0000","last_unstale":"2023-07-14T12:00:07.329782+0000","last_undegraded":"2023-07-14T12:00:07.329782+0000","last_fullsized":"2023-07-14T12:00:07.329782+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:49:01.837874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.807313+0000","last_change":"2023-07-14T11:59:47.916411+0000","last_active":"2023-07-14T12:00:06.807313+0000","last_peered":"2023-07-14T12:00:06.807313+0000","last_clean":"2023-07-14T12:00:06.807313+0000","last_became_active":"2023-07-14T11:59:47.915877+0000","last_became_peered":"2023-07-14T11:59:47.915877+0000","last_unstale":"2023-07-14T12:00:06.807313+0000","last_undegraded":"2023-07-14T12:00:06.807313+0000","last_fullsized":"2023-07-14T12:00:06.807313+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:08:05.471264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:13.463576+0000","last_change":"2023-07-14T11:59:42.769080+0000","last_active":"2023-07-14T12:00:13.463576+0000","last_peered":"2023-07-14T12:00:13.463576+0000","last_clean":"2023-07-14T12:00:13.463576+0000","last_became_active":"2023-07-14T11:59:42.768782+0000","last_became_peered":"2023-07-14T11:59:42.768782+0000","last_unstale":"2023-07-14T12:00:13.463576+0000","last_undegraded":"2023-07-14T12:00:13.463576+0000","last_fullsized":"2023-07-14T12:00:13.463576+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:31:48.128002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.806874+0000","last_change":"2023-07-14T11:59:43.460284+0000","last_active":"2023-07-14T12:00:06.806874+0000","last_peered":"2023-07-14T12:00:06.806874+0000","last_clean":"2023-07-14T12:00:06.806874+0000","last_became_active":"2023-07-14T11:59:43.459979+0000","last_became_peered":"2023-07-14T11:59:43.459979+0000","last_unstale":"2023-07-14T12:00:06.806874+0000","last_undegraded":"2023-07-14T12:00:06.806874+0000","last_fullsized":"2023-07-14T12:00:06.806874+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:25:15.513703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-14T12:01:06.808085+0000","last_change":"2023-07-14T11:59:47.782359+0000","last_active":"2023-07-14T12:01:06.808085+0000","last_peered":"2023-07-14T12:01:06.808085+0000","last_clean":"2023-07-14T12:01:06.808085+0000","last_became_active":"2023-07-14T11:59:47.780484+0000","last_became_peered":"2023-07-14T11:59:47.780484+0000","last_unstale":"2023-07-14T12:01:06.808085+0000","last_undegraded":"2023-07-14T12:01:06.808085+0000","last_fullsized":"2023-07-14T12:01:06.808085+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:18:30.825171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:16.495085+0000","last_change":"2023-07-14T11:59:42.756823+0000","last_active":"2023-07-14T12:00:16.495085+0000","last_peered":"2023-07-14T12:00:16.495085+0000","last_clean":"2023-07-14T12:00:16.495085+0000","last_became_active":"2023-07-14T11:59:42.756412+0000","last_became_peered":"2023-07-14T11:59:42.756412+0000","last_unstale":"2023-07-14T12:00:16.495085+0000","last_undegraded":"2023-07-14T12:00:16.495085+0000","last_fullsized":"2023-07-14T12:00:16.495085+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:45:25.674809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.811156+0000","last_change":"2023-07-14T11:59:43.469151+0000","last_active":"2023-07-14T12:00:06.811156+0000","last_peered":"2023-07-14T12:00:06.811156+0000","last_clean":"2023-07-14T12:00:06.811156+0000","last_became_active":"2023-07-14T11:59:43.468835+0000","last_became_peered":"2023-07-14T11:59:43.468835+0000","last_unstale":"2023-07-14T12:00:06.811156+0000","last_undegraded":"2023-07-14T12:00:06.811156+0000","last_fullsized":"2023-07-14T12:00:06.811156+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:01:00.517226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.807655+0000","last_change":"2023-07-14T11:59:47.782663+0000","last_active":"2023-07-14T12:00:06.807655+0000","last_peered":"2023-07-14T12:00:06.807655+0000","last_clean":"2023-07-14T12:00:06.807655+0000","last_became_active":"2023-07-14T11:59:47.782175+0000","last_became_peered":"2023-07-14T11:59:47.782175+0000","last_unstale":"2023-07-14T12:00:06.807655+0000","last_undegraded":"2023-07-14T12:00:06.807655+0000","last_fullsized":"2023-07-14T12:00:06.807655+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:43:48.229057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:16.994892+0000","last_change":"2023-07-14T11:59:42.759809+0000","last_active":"2023-07-14T12:00:16.994892+0000","last_peered":"2023-07-14T12:00:16.994892+0000","last_clean":"2023-07-14T12:00:16.994892+0000","last_became_active":"2023-07-14T11:59:42.759493+0000","last_became_peered":"2023-07-14T11:59:42.759493+0000","last_unstale":"2023-07-14T12:00:16.994892+0000","last_undegraded":"2023-07-14T12:00:16.994892+0000","last_fullsized":"2023-07-14T12:00:16.994892+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:21:22.391310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.810840+0000","last_change":"2023-07-14T11:59:43.465308+0000","last_active":"2023-07-14T12:00:06.810840+0000","last_peered":"2023-07-14T12:00:06.810840+0000","last_clean":"2023-07-14T12:00:06.810840+0000","last_became_active":"2023-07-14T11:59:43.464995+0000","last_became_peered":"2023-07-14T11:59:43.464995+0000","last_unstale":"2023-07-14T12:00:06.810840+0000","last_undegraded":"2023-07-14T12:00:06.810840+0000","last_fullsized":"2023-07-14T12:00:06.810840+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:22:41.954399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.807244+0000","last_change":"2023-07-14T11:59:47.916435+0000","last_active":"2023-07-14T12:00:06.807244+0000","last_peered":"2023-07-14T12:00:06.807244+0000","last_clean":"2023-07-14T12:00:06.807244+0000","last_became_active":"2023-07-14T11:59:47.915894+0000","last_became_peered":"2023-07-14T11:59:47.915894+0000","last_unstale":"2023-07-14T12:00:06.807244+0000","last_undegraded":"2023-07-14T12:00:06.807244+0000","last_fullsized":"2023-07-14T12:00:06.807244+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:15:10.755288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:13.466102+0000","last_change":"2023-07-14T11:59:42.760422+0000","last_active":"2023-07-14T12:00:13.466102+0000","last_peered":"2023-07-14T12:00:13.466102+0000","last_clean":"2023-07-14T12:00:13.466102+0000","last_became_active":"2023-07-14T11:59:42.758548+0000","last_became_peered":"2023-07-14T11:59:42.758548+0000","last_unstale":"2023-07-14T12:00:13.466102+0000","last_undegraded":"2023-07-14T12:00:13.466102+0000","last_fullsized":"2023-07-14T12:00:13.466102+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:00:42.799391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.328110+0000","last_change":"2023-07-14T11:59:43.455296+0000","last_active":"2023-07-14T12:00:07.328110+0000","last_peered":"2023-07-14T12:00:07.328110+0000","last_clean":"2023-07-14T12:00:07.328110+0000","last_became_active":"2023-07-14T11:59:43.455003+0000","last_became_peered":"2023-07-14T11:59:43.455003+0000","last_unstale":"2023-07-14T12:00:07.328110+0000","last_undegraded":"2023-07-14T12:00:07.328110+0000","last_fullsized":"2023-07-14T12:00:07.328110+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T13:43:29.555694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805023+0000","last_change":"2023-07-14T11:59:47.779262+0000","last_active":"2023-07-14T12:00:06.805023+0000","last_peered":"2023-07-14T12:00:06.805023+0000","last_clean":"2023-07-14T12:00:06.805023+0000","last_became_active":"2023-07-14T11:59:47.778863+0000","last_became_peered":"2023-07-14T11:59:47.778863+0000","last_unstale":"2023-07-14T12:00:06.805023+0000","last_undegraded":"2023-07-14T12:00:06.805023+0000","last_fullsized":"2023-07-14T12:00:06.805023+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:11:16.752296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'5","reported_seq":85,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-14T12:01:06.814717+0000","last_change":"2023-07-14T11:59:42.768252+0000","last_active":"2023-07-14T12:01:06.814717+0000","last_peered":"2023-07-14T12:01:06.814717+0000","last_clean":"2023-07-14T12:01:06.814717+0000","last_became_active":"2023-07-14T11:59:42.767956+0000","last_became_peered":"2023-07-14T11:59:42.767956+0000","last_unstale":"2023-07-14T12:01:06.814717+0000","last_undegraded":"2023-07-14T12:01:06.814717+0000","last_fullsized":"2023-07-14T12:01:06.814717+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:28:56.297775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":52,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.328734+0000","last_change":"2023-07-14T11:59:43.461899+0000","last_active":"2023-07-14T12:00:07.328734+0000","last_peered":"2023-07-14T12:00:07.328734+0000","last_clean":"2023-07-14T12:00:07.328734+0000","last_became_active":"2023-07-14T11:59:43.461535+0000","last_became_peered":"2023-07-14T11:59:43.461535+0000","last_unstale":"2023-07-14T12:00:07.328734+0000","last_undegraded":"2023-07-14T12:00:07.328734+0000","last_fullsized":"2023-07-14T12:00:07.328734+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:43:24.603371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.812720+0000","last_change":"2023-07-14T11:59:47.782964+0000","last_active":"2023-07-14T12:00:06.812720+0000","last_peered":"2023-07-14T12:00:06.812720+0000","last_clean":"2023-07-14T12:00:06.812720+0000","last_became_active":"2023-07-14T11:59:47.782563+0000","last_became_peered":"2023-07-14T11:59:47.782563+0000","last_unstale":"2023-07-14T12:00:06.812720+0000","last_undegraded":"2023-07-14T12:00:06.812720+0000","last_fullsized":"2023-07-14T12:00:06.812720+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:27:49.195128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:16.494770+0000","last_change":"2023-07-14T11:59:42.751355+0000","last_active":"2023-07-14T12:00:16.494770+0000","last_peered":"2023-07-14T12:00:16.494770+0000","last_clean":"2023-07-14T12:00:16.494770+0000","last_became_active":"2023-07-14T11:59:42.751063+0000","last_became_peered":"2023-07-14T11:59:42.751063+0000","last_unstale":"2023-07-14T12:00:16.494770+0000","last_undegraded":"2023-07-14T12:00:16.494770+0000","last_fullsized":"2023-07-14T12:00:16.494770+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:28:15.966689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.328252+0000","last_change":"2023-07-14T11:59:43.465891+0000","last_active":"2023-07-14T12:00:07.328252+0000","last_peered":"2023-07-14T12:00:07.328252+0000","last_clean":"2023-07-14T12:00:07.328252+0000","last_became_active":"2023-07-14T11:59:43.465575+0000","last_became_peered":"2023-07-14T11:59:43.465575+0000","last_unstale":"2023-07-14T12:00:07.328252+0000","last_undegraded":"2023-07-14T12:00:07.328252+0000","last_fullsized":"2023-07-14T12:00:07.328252+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:43:52.704477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.809069+0000","last_change":"2023-07-14T11:59:47.777547+0000","last_active":"2023-07-14T12:00:06.809069+0000","last_peered":"2023-07-14T12:00:06.809069+0000","last_clean":"2023-07-14T12:00:06.809069+0000","last_became_active":"2023-07-14T11:59:47.777150+0000","last_became_peered":"2023-07-14T11:59:47.777150+0000","last_unstale":"2023-07-14T12:00:06.809069+0000","last_undegraded":"2023-07-14T12:00:06.809069+0000","last_fullsized":"2023-07-14T12:00:06.809069+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:46.757401+0000","last_clean_scrub_stamp":"2023-07-14T11:59:46.757401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:21:35.638026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:16.494997+0000","last_change":"2023-07-14T11:59:42.761236+0000","last_active":"2023-07-14T12:00:16.494997+0000","last_peered":"2023-07-14T12:00:16.494997+0000","last_clean":"2023-07-14T12:00:16.494997+0000","last_became_active":"2023-07-14T11:59:42.760893+0000","last_became_peered":"2023-07-14T11:59:42.760893+0000","last_unstale":"2023-07-14T12:00:16.494997+0000","last_undegraded":"2023-07-14T12:00:16.494997+0000","last_fullsized":"2023-07-14T12:00:16.494997+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:41.724226+0000","last_clean_scrub_stamp":"2023-07-14T11:59:41.724226+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:11:17.477118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:07.328670+0000","last_change":"2023-07-14T11:59:43.462518+0000","last_active":"2023-07-14T12:00:07.328670+0000","last_peered":"2023-07-14T12:00:07.328670+0000","last_clean":"2023-07-14T12:00:07.328670+0000","last_became_active":"2023-07-14T11:59:43.462178+0000","last_became_peered":"2023-07-14T11:59:43.462178+0000","last_unstale":"2023-07-14T12:00:07.328670+0000","last_undegraded":"2023-07-14T12:00:07.328670+0000","last_fullsized":"2023-07-14T12:00:07.328670+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:09:38.398771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.811302+0000","last_change":"2023-07-14T11:59:43.474142+0000","last_active":"2023-07-14T12:00:06.811302+0000","last_peered":"2023-07-14T12:00:06.811302+0000","last_clean":"2023-07-14T12:00:06.811302+0000","last_became_active":"2023-07-14T11:59:43.473863+0000","last_became_peered":"2023-07-14T11:59:43.473863+0000","last_unstale":"2023-07-14T12:00:06.811302+0000","last_undegraded":"2023-07-14T12:00:06.811302+0000","last_fullsized":"2023-07-14T12:00:06.811302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:34:40.849796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-14T12:01:06.814397+0000","last_change":"2023-07-14T11:59:43.462193+0000","last_active":"2023-07-14T12:01:06.814397+0000","last_peered":"2023-07-14T12:01:06.814397+0000","last_clean":"2023-07-14T12:01:06.814397+0000","last_became_active":"2023-07-14T11:59:43.461390+0000","last_became_peered":"2023-07-14T11:59:43.461390+0000","last_unstale":"2023-07-14T12:01:06.814397+0000","last_undegraded":"2023-07-14T12:01:06.814397+0000","last_fullsized":"2023-07-14T12:01:06.814397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:11:05.240351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-14T12:01:06.814132+0000","last_change":"2023-07-14T11:59:43.467561+0000","last_active":"2023-07-14T12:01:06.814132+0000","last_peered":"2023-07-14T12:01:06.814132+0000","last_clean":"2023-07-14T12:01:06.814132+0000","last_became_active":"2023-07-14T11:59:43.467256+0000","last_became_peered":"2023-07-14T11:59:43.467256+0000","last_unstale":"2023-07-14T12:01:06.814132+0000","last_undegraded":"2023-07-14T12:01:06.814132+0000","last_fullsized":"2023-07-14T12:01:06.814132+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:40:11.167658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.813046+0000","last_change":"2023-07-14T11:59:43.472405+0000","last_active":"2023-07-14T12:00:06.813046+0000","last_peered":"2023-07-14T12:00:06.813046+0000","last_clean":"2023-07-14T12:00:06.813046+0000","last_became_active":"2023-07-14T11:59:43.471863+0000","last_became_peered":"2023-07-14T11:59:43.471863+0000","last_unstale":"2023-07-14T12:00:06.813046+0000","last_undegraded":"2023-07-14T12:00:06.813046+0000","last_fullsized":"2023-07-14T12:00:06.813046+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:22:54.987390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808865+0000","last_change":"2023-07-14T11:59:43.469966+0000","last_active":"2023-07-14T12:00:06.808865+0000","last_peered":"2023-07-14T12:00:06.808865+0000","last_clean":"2023-07-14T12:00:06.808865+0000","last_became_active":"2023-07-14T11:59:43.469540+0000","last_became_peered":"2023-07-14T11:59:43.469540+0000","last_unstale":"2023-07-14T12:00:06.808865+0000","last_undegraded":"2023-07-14T12:00:06.808865+0000","last_fullsized":"2023-07-14T12:00:06.808865+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T15:19:04.529881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.812535+0000","last_change":"2023-07-14T11:59:43.465032+0000","last_active":"2023-07-14T12:00:06.812535+0000","last_peered":"2023-07-14T12:00:06.812535+0000","last_clean":"2023-07-14T12:00:06.812535+0000","last_became_active":"2023-07-14T11:59:43.464689+0000","last_became_peered":"2023-07-14T11:59:43.464689+0000","last_unstale":"2023-07-14T12:00:06.812535+0000","last_undegraded":"2023-07-14T12:00:06.812535+0000","last_fullsized":"2023-07-14T12:00:06.812535+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:15:12.845214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-14T12:01:06.815119+0000","last_change":"2023-07-14T11:59:43.463042+0000","last_active":"2023-07-14T12:01:06.815119+0000","last_peered":"2023-07-14T12:01:06.815119+0000","last_clean":"2023-07-14T12:01:06.815119+0000","last_became_active":"2023-07-14T11:59:43.462665+0000","last_became_peered":"2023-07-14T11:59:43.462665+0000","last_unstale":"2023-07-14T12:01:06.815119+0000","last_undegraded":"2023-07-14T12:01:06.815119+0000","last_fullsized":"2023-07-14T12:01:06.815119+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:54:34.295388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.816236+0000","last_change":"2023-07-14T12:00:06.816236+0000","last_active":"2023-07-14T12:00:06.816236+0000","last_peered":"2023-07-14T12:00:06.816236+0000","last_clean":"2023-07-14T12:00:06.816236+0000","last_became_active":"2023-07-14T12:00:06.815733+0000","last_became_peered":"2023-07-14T12:00:06.815733+0000","last_unstale":"2023-07-14T12:00:06.816236+0000","last_undegraded":"2023-07-14T12:00:06.816236+0000","last_fullsized":"2023-07-14T12:00:06.816236+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:14:57.114402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.809719+0000","last_change":"2023-07-14T11:59:43.470256+0000","last_active":"2023-07-14T12:00:06.809719+0000","last_peered":"2023-07-14T12:00:06.809719+0000","last_clean":"2023-07-14T12:00:06.809719+0000","last_became_active":"2023-07-14T11:59:43.469861+0000","last_became_peered":"2023-07-14T11:59:43.469861+0000","last_unstale":"2023-07-14T12:00:06.809719+0000","last_undegraded":"2023-07-14T12:00:06.809719+0000","last_fullsized":"2023-07-14T12:00:06.809719+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:54:04.221673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.811456+0000","last_change":"2023-07-14T11:59:43.458009+0000","last_active":"2023-07-14T12:00:06.811456+0000","last_peered":"2023-07-14T12:00:06.811456+0000","last_clean":"2023-07-14T12:00:06.811456+0000","last_became_active":"2023-07-14T11:59:43.457657+0000","last_became_peered":"2023-07-14T11:59:43.457657+0000","last_unstale":"2023-07-14T12:00:06.811456+0000","last_undegraded":"2023-07-14T12:00:06.811456+0000","last_fullsized":"2023-07-14T12:00:06.811456+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:14:41.854547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.805358+0000","last_change":"2023-07-14T11:59:43.457568+0000","last_active":"2023-07-14T12:00:06.805358+0000","last_peered":"2023-07-14T12:00:06.805358+0000","last_clean":"2023-07-14T12:00:06.805358+0000","last_became_active":"2023-07-14T11:59:43.457203+0000","last_became_peered":"2023-07-14T11:59:43.457203+0000","last_unstale":"2023-07-14T12:00:06.805358+0000","last_undegraded":"2023-07-14T12:00:06.805358+0000","last_fullsized":"2023-07-14T12:00:06.805358+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T12:14:37.939509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.808045+0000","last_change":"2023-07-14T11:59:43.470840+0000","last_active":"2023-07-14T12:00:06.808045+0000","last_peered":"2023-07-14T12:00:06.808045+0000","last_clean":"2023-07-14T12:00:06.808045+0000","last_became_active":"2023-07-14T11:59:43.470531+0000","last_became_peered":"2023-07-14T11:59:43.470531+0000","last_unstale":"2023-07-14T12:00:06.808045+0000","last_undegraded":"2023-07-14T12:00:06.808045+0000","last_fullsized":"2023-07-14T12:00:06.808045+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T14:49:50.862808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-14T12:00:06.817057+0000","last_change":"2023-07-14T12:00:06.817057+0000","last_active":"2023-07-14T12:00:06.817057+0000","last_peered":"2023-07-14T12:00:06.817057+0000","last_clean":"2023-07-14T12:00:06.817057+0000","last_became_active":"2023-07-14T12:00:06.816680+0000","last_became_peered":"2023-07-14T12:00:06.816680+0000","last_unstale":"2023-07-14T12:00:06.817057+0000","last_undegraded":"2023-07-14T12:00:06.817057+0000","last_fullsized":"2023-07-14T12:00:06.817057+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T11:59:42.732294+0000","last_clean_scrub_stamp":"2023-07-14T11:59:42.732294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:22:01.363770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284390,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":10024,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":513,"num_read_kb":276,"num_write":209,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":30072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416413,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":983040,"data_stored":487359,"data_compressed":17563,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 12:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.58499999999999996,"15min":0.59099999999999997},"min":{"1min":0.34599999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.54500000000000004,"15min":0.55100000000000005},"min":{"1min":0.38300000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.72499999999999998,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.64000000000000001}]},{"osd":1,"last update":"Fri Jul 14 12:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82499999999999996,"5min":1.048,"15min":1.048},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.60999999999999999,"15min":0.60199999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.53000000000000003}]},{"osd":2,"last update":"Fri Jul 14 12:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.499,"15min":0.498},"min":{"1min":0.317,"5min":0.253,"15min":0.253},"max":{"1min":0.755,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.502,"15min":0.496},"min":{"1min":0.34100000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.78500000000000003,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.61699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":4,"last update":"Fri Jul 14 12:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.68600000000000005,"15min":0.67600000000000005},"min":{"1min":0.5,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.879,"5min":1.149,"15min":1.149},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.64400000000000002,"15min":0.64100000000000001},"min":{"1min":0.42199999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81299999999999994,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.59299999999999997}]},{"osd":5,"last update":"Fri Jul 14 12:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56100000000000005,"15min":0.56299999999999994},"min":{"1min":0.40300000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.67900000000000005,"5min":1.169,"15min":1.169},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.64400000000000002,"15min":0.64200000000000002},"min":{"1min":0.42199999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.88500000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.748}]},{"osd":6,"last update":"Fri Jul 14 12:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.70799999999999996,"15min":0.71999999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.95999999999999996,"5min":1.21,"15min":1.21},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.73499999999999999,"15min":0.753},"min":{"1min":0.46100000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.79800000000000004,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":0.70199999999999996}]},{"osd":7,"last update":"Fri Jul 14 12:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.75700000000000001,"15min":0.754},"min":{"1min":0.56200000000000006,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.96299999999999997,"5min":1.327,"15min":1.327},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.76400000000000001,"15min":0.76300000000000001},"min":{"1min":0.57199999999999995,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.93100000000000005,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.81299999999999994}]},{"osd":8,"last update":"Fri Jul 14 12:00:46 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.86499999999999999}]},{"osd":9,"last update":"Fri Jul 14 12:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.81799999999999995,"15min":0.80700000000000005},"min":{"1min":0.52600000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":3.9089999999999998,"5min":3.9089999999999998,"15min":3.9089999999999998},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.81699999999999995,"15min":0.81499999999999995},"min":{"1min":0.63300000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.98299999999999998,"5min":1.385,"15min":1.385},"last":0.88}]},{"osd":10,"last update":"Fri Jul 14 12:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.60799999999999998,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.995,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.79800000000000004,"15min":0.80300000000000005},"min":{"1min":0.59699999999999998,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.91600000000000004,"5min":2.1019999999999999,"15min":2.1019999999999999},"last":0.77000000000000002}]}]},{"osd":10,"up_from":72,"seq":309237645344,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":1048,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1073152,"data_stored":1081734,"data_compressed":53729,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":3.375,"5min":3.375,"15min":3.375},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.67200000000000004}]},{"osd":1,"last update":"Fri Jul 14 12:00:37 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.496,"15min":0.504},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.63400000000000001,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.30199999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.65700000000000003}]},{"osd":2,"last update":"Fri Jul 14 12:00:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.60599999999999998,"15min":0.60699999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.61099999999999999,"15min":0.61199999999999999},"min":{"1min":0.42799999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.69199999999999995}]},{"osd":3,"last update":"Fri Jul 14 12:00:37 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.70399999999999996,"15min":0.67100000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":7.1159999999999997,"5min":7.1159999999999997,"15min":7.1159999999999997},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.67400000000000004,"15min":0.68300000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.80100000000000005,"5min":0.996,"15min":0.996},"last":0.56799999999999995}]},{"osd":4,"last update":"Fri Jul 14 12:00:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.56899999999999995,"15min":0.57499999999999996},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77400000000000002,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.38100000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.77400000000000002,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.77200000000000002}]},{"osd":5,"last update":"Fri Jul 14 12:00:37 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.65000000000000002,"15min":0.64800000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.749},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.70199999999999996,"15min":0.71399999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.79700000000000004,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.82499999999999996}]},{"osd":6,"last update":"Fri Jul 14 12:00:37 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.80600000000000005,"15min":0.78200000000000003},"min":{"1min":0.47999999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":7.0190000000000001,"5min":7.0190000000000001,"15min":7.0190000000000001},"last":0.88},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.82499999999999996,"15min":0.80500000000000005},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":5.5110000000000001,"5min":5.5110000000000001,"15min":5.5110000000000001},"last":0.76100000000000001}]},{"osd":7,"last update":"Fri Jul 14 11:59:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.60899999999999999}]},{"osd":8,"last update":"Fri Jul 14 12:00:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.77100000000000002,"15min":0.78400000000000003},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.76300000000000001,"15min":0.77000000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.96699999999999997,"5min":1.054,"15min":1.054},"last":0.95499999999999996}]},{"osd":9,"last update":"Fri Jul 14 12:00:37 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.80800000000000005,"15min":0.81999999999999995},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.90200000000000002,"5min":1.075,"15min":1.075},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.84399999999999997,"15min":0.82599999999999996},"min":{"1min":0.59399999999999997,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":5.367,"5min":5.367,"15min":5.367},"last":0.89600000000000002}]},{"osd":11,"last update":"Fri Jul 14 12:00:58 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.76400000000000001,"15min":0.77000000000000002},"min":{"1min":0.52100000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.86599999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.76800000000000002,"15min":0.77300000000000002},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.95199999999999996,"5min":1.05,"15min":1.05},"last":0.93700000000000006}]}]},{"osd":9,"up_from":67,"seq":287762808870,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":1007616,"data_stored":487337,"data_compressed":17563,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 12:00:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.63,"15min":0.64400000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.252},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.64100000000000001,"15min":0.65000000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.312}]},{"osd":1,"last update":"Fri Jul 14 12:00:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.755,"15min":0.76800000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.877,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.80200000000000005,"15min":0.81599999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.52300000000000002}]},{"osd":2,"last update":"Fri Jul 14 12:00:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.76800000000000002,"15min":0.77700000000000002},"min":{"1min":0.54400000000000004,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.99199999999999999,"5min":1.147,"15min":1.147},"last":0.371},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.79200000000000004,"15min":0.81200000000000006},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.91100000000000003,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.46200000000000002}]},{"osd":3,"last update":"Fri Jul 14 12:00:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.73599999999999999,"15min":0.75900000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.79700000000000004,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.437},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.75900000000000001,"15min":0.78000000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0980000000000001,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.32600000000000001}]},{"osd":4,"last update":"Fri Jul 14 12:00:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.871,"15min":0.89700000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.95599999999999996,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.94999999999999996,"15min":0.98199999999999998},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.012,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":0.51100000000000001}]},{"osd":5,"last update":"Fri Jul 14 12:00:16 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.88,"15min":0.90200000000000002},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.94799999999999995,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.93200000000000005,"15min":0.95599999999999996},"min":{"1min":0.58599999999999997,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0589999999999999,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.42499999999999999}]},{"osd":6,"last update":"Fri Jul 14 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":1.006}]},{"osd":7,"last update":"Fri Jul 14 12:00:16 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.995,"15min":1.024},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.05,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.82399999999999995,"5min":1.008,"15min":1.0389999999999999},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.1240000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.45200000000000001}]},{"osd":8,"last update":"Fri Jul 14 12:00:16 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.96899999999999997,"15min":0.98699999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.306,"5min":1.391,"15min":1.391},"last":0.495},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.99299999999999999,"15min":1.0169999999999999},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.0800000000000001,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.61199999999999999}]},{"osd":10,"last update":"Fri Jul 14 12:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.94099999999999995,"15min":0.94699999999999995},"min":{"1min":0.627,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.2130000000000001,"5min":1.238,"15min":1.238},"last":0.629},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.93300000000000005,"15min":0.94099999999999995},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.1950000000000001,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.48299999999999998}]},{"osd":11,"last update":"Fri Jul 14 12:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.50600000000000001}]}]},{"osd":8,"up_from":59,"seq":253403070507,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":1003520,"data_stored":487715,"data_compressed":17563,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 12:00:51 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.67800000000000005,"15min":0.63400000000000001},"min":{"1min":0.434,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":8.0559999999999992,"5min":8.0559999999999992,"15min":8.0559999999999992},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.61299999999999999,"15min":0.59699999999999998},"min":{"1min":0.40100000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":4.1600000000000001,"5min":4.1600000000000001,"15min":4.1600000000000001},"last":0.55900000000000005}]},{"osd":1,"last update":"Fri Jul 14 12:00:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.60499999999999998,"15min":0.58699999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.63700000000000001,"15min":0.63},"min":{"1min":0.53000000000000003,"5min":0.376,"15min":0.376},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.69299999999999995}]},{"osd":2,"last update":"Fri Jul 14 12:00:51 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.47999999999999998,"15min":0.47399999999999998},"min":{"1min":0.38,"5min":0.309,"15min":0.309},"max":{"1min":0.66000000000000003,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.51000000000000001,"15min":0.51800000000000002},"min":{"1min":0.30599999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.65600000000000003}]},{"osd":3,"last update":"Fri Jul 14 12:00:51 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":0.71399999999999997,"15min":0.67000000000000004},"min":{"1min":0.41099999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":8.6929999999999996,"5min":8.6929999999999996,"15min":8.6929999999999996},"last":0.70699999999999996},{"interface":"front","average":{"1min":1.0589999999999999,"5min":0.72999999999999998,"15min":0.68100000000000005},"min":{"1min":0.45300000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":8.2520000000000007,"5min":8.2520000000000007,"15min":8.2520000000000007},"last":0.626}]},{"osd":4,"last update":"Fri Jul 14 12:00:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.622,"15min":0.622},"min":{"1min":0.40600000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81699999999999995,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.66900000000000004,"15min":0.65200000000000002},"min":{"1min":0.53600000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.72599999999999998}]},{"osd":5,"last update":"Fri Jul 14 12:00:51 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.61799999999999999,"15min":0.621},"min":{"1min":0.35399999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.83699999999999997,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.59499999999999997,"15min":0.59099999999999997},"min":{"1min":0.40100000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.73999999999999999}]},{"osd":6,"last update":"Fri Jul 14 12:00:51 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.747,"15min":0.751},"min":{"1min":0.56100000000000005,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.80400000000000005,"15min":0.79900000000000004},"min":{"1min":0.56399999999999995,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":2.7429999999999999,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":0.755}]},{"osd":7,"last update":"Fri Jul 14 12:00:51 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.75800000000000001,"15min":0.755},"min":{"1min":0.47199999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.745,"15min":0.749},"min":{"1min":0.59399999999999997,"5min":0.435,"15min":0.435},"max":{"1min":1.0600000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.86899999999999999}]},{"osd":9,"last update":"Fri Jul 14 12:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.78500000000000003,"15min":0.77900000000000003},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.80400000000000005,"15min":0.79800000000000004},"min":{"1min":0.59299999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.0269999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.90500000000000003}]},{"osd":10,"last update":"Fri Jul 14 12:00:37 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.78700000000000003,"15min":0.78000000000000003},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.77800000000000002,"15min":0.76800000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.88800000000000001}]},{"osd":11,"last update":"Fri Jul 14 12:00:45 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.82799999999999996}]}]},{"osd":7,"up_from":52,"seq":223338299439,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":1048576,"data_stored":1079250,"data_compressed":53729,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.443,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.88300000000000001,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59699999999999998,"15min":0.59599999999999997},"min":{"1min":0.50700000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.495}]},{"osd":1,"last update":"Fri Jul 14 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42299999999999999,"15min":0.41599999999999998},"min":{"1min":0.317,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.69699999999999995,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44800000000000001},"min":{"1min":0.312,"5min":0.249,"15min":0.249},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.50900000000000001}]},{"osd":2,"last update":"Fri Jul 14 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59399999999999997,"15min":0.59699999999999998},"min":{"1min":0.39100000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73199999999999998,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.63700000000000001,"15min":0.621},"min":{"1min":0.51700000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.68500000000000005}]},{"osd":3,"last update":"Fri Jul 14 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.623,"15min":0.626},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85599999999999998,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.64300000000000002,"15min":0.65200000000000002},"min":{"1min":0.47299999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.75900000000000001,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.77700000000000002}]},{"osd":4,"last update":"Fri Jul 14 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52000000000000002,"15min":0.504},"min":{"1min":0.42699999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.71099999999999997,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.52300000000000002,"15min":0.52900000000000003},"min":{"1min":0.38300000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.80600000000000005,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.54600000000000004}]},{"osd":5,"last update":"Fri Jul 14 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.64400000000000002,"15min":0.63800000000000001},"min":{"1min":0.54800000000000004,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.79900000000000004,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71599999999999997,"15min":0.72099999999999997},"min":{"1min":0.54900000000000004,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.82399999999999995,"5min":1.341,"15min":1.341},"last":0.76200000000000001}]},{"osd":6,"last update":"Fri Jul 14 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.71199999999999997,"15min":0.71699999999999997},"min":{"1min":0.53300000000000003,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.81899999999999995,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.68500000000000005,"15min":0.69299999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.80600000000000005,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.85999999999999999}]},{"osd":8,"last update":"Fri Jul 14 12:00:51 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.72999999999999998,"15min":0.70599999999999996},"min":{"1min":0.46999999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.96399999999999997,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.71499999999999997,"15min":0.70399999999999996},"min":{"1min":0.54200000000000004,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93000000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.89000000000000001}]},{"osd":9,"last update":"Fri Jul 14 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.58199999999999996,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.93500000000000005,"5min":0.995,"15min":0.995},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.77600000000000002,"15min":0.77900000000000003},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.83499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.91900000000000004}]},{"osd":10,"last update":"Fri Jul 14 12:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67800000000000005,"15min":0.67900000000000005},"min":{"1min":0.46700000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.75600000000000001,"15min":0.76200000000000001},"min":{"1min":0.51000000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.90400000000000003,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.73099999999999998}]},{"osd":11,"last update":"Fri Jul 14 12:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.878}]}]},{"osd":6,"up_from":47,"seq":201863462963,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":987136,"data_stored":486889,"data_compressed":17563,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 12:00:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.48699999999999999,"15min":0.47299999999999998},"min":{"1min":0.39100000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.502},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.60599999999999998,"15min":0.59799999999999998},"min":{"1min":0.46600000000000003,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.67800000000000005}]},{"osd":1,"last update":"Fri Jul 14 12:00:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.65600000000000003,"15min":0.64600000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.72699999999999998,"15min":0.72199999999999998},"min":{"1min":0.51200000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.621}]},{"osd":2,"last update":"Fri Jul 14 12:00:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.65500000000000003,"15min":0.64500000000000002},"min":{"1min":0.48999999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.93799999999999994,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.70099999999999996,"15min":0.68999999999999995},"min":{"1min":0.41399999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0049999999999999,"5min":1.155,"15min":1.155},"last":0.80800000000000005}]},{"osd":3,"last update":"Fri Jul 14 12:00:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.61399999999999999,"15min":0.59999999999999998},"min":{"1min":0.46200000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.93600000000000005,"5min":1.107,"15min":1.107},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.70899999999999996,"15min":0.69699999999999995},"min":{"1min":0.57499999999999996,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91500000000000004,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.72099999999999997}]},{"osd":4,"last update":"Fri Jul 14 12:00:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.68500000000000005,"15min":0.67300000000000004},"min":{"1min":0.441,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.754,"15min":0.73999999999999999},"min":{"1min":0.57199999999999995,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.65800000000000003}]},{"osd":5,"last update":"Fri Jul 14 12:00:06 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.79900000000000004,"15min":0.79300000000000004},"min":{"1min":0.67600000000000005,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1000000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.745,"15min":0.73399999999999999},"min":{"1min":0.55000000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.90400000000000003}]},{"osd":7,"last update":"Fri Jul 14 12:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.80100000000000005,"15min":0.76200000000000001},"min":{"1min":0.59299999999999997,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.0920000000000001,"5min":0.81299999999999994,"15min":0.74099999999999999},"min":{"1min":0.57399999999999995,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":6.4400000000000004,"5min":6.4400000000000004,"15min":6.4400000000000004},"last":0.82199999999999995}]},{"osd":8,"last update":"Fri Jul 14 12:00:48 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.83399999999999996,"15min":0.81100000000000005},"min":{"1min":0.68500000000000005,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":1.0600000000000001},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.85099999999999998,"15min":0.82199999999999995},"min":{"1min":0.67200000000000004,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.873}]},{"osd":9,"last update":"Fri Jul 14 12:00:13 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.93600000000000005,"15min":0.93200000000000005},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.92900000000000005,"15min":0.92200000000000004},"min":{"1min":0.68000000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":1.0169999999999999}]},{"osd":10,"last update":"Fri Jul 14 12:00:37 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.01,"5min":1.012,"15min":1.012},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.84899999999999998}]},{"osd":11,"last update":"Fri Jul 14 12:00:48 2023","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.046,"15min":1.046},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.93899999999999995},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":1.0840000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":995328,"data_stored":491713,"data_compressed":17563,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54800000000000004,"15min":0.52800000000000002},"min":{"1min":0.30199999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.83299999999999996,"5min":1.153,"15min":1.153},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.60799999999999998,"15min":0.72499999999999998},"min":{"1min":0.35099999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.872,"5min":5.0940000000000003,"15min":5.0940000000000003},"last":0.56899999999999995}]},{"osd":2,"last update":"Fri Jul 14 12:00:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.55500000000000005,"15min":0.50700000000000001},"min":{"1min":0.32600000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.82099999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.624,"15min":0.69699999999999995},"min":{"1min":0.434,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.84899999999999998,"5min":5.0570000000000004,"15min":5.0570000000000004},"last":0.67400000000000004}]},{"osd":3,"last update":"Fri Jul 14 12:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60999999999999999,"15min":0.59799999999999998},"min":{"1min":0.42099999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.72299999999999998,"5min":1.054,"15min":1.054},"last":0.624},{"interface":"front","average":{"1min":0.622,"5min":0.61899999999999999,"15min":0.61299999999999999},"min":{"1min":0.438,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.77900000000000003,"5min":1.006,"15min":1.006},"last":0.748}]},{"osd":4,"last update":"Fri Jul 14 12:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.45800000000000002,"15min":0.42799999999999999},"min":{"1min":0.36799999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.69099999999999995,"5min":1,"15min":1},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.48099999999999998,"15min":0.45800000000000002},"min":{"1min":0.33700000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.997}]},{"osd":5,"last update":"Fri Jul 14 12:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.64200000000000002,"15min":0.59599999999999997},"min":{"1min":0.48799999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.61499999999999999,"15min":0.56699999999999995},"min":{"1min":0.53300000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.94999999999999996,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.78400000000000003}]},{"osd":6,"last update":"Fri Jul 14 12:00:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.65700000000000003,"15min":0.63900000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.749,"5min":0.69399999999999995,"15min":0.67500000000000004},"min":{"1min":0.53800000000000003,"5min":0.44,"15min":0.44},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":1.0840000000000001}]},{"osd":7,"last update":"Fri Jul 14 12:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.55600000000000005,"15min":0.53600000000000003},"min":{"1min":0.42699999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.64600000000000002,"15min":0.60399999999999998},"min":{"1min":0.47899999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":1.0489999999999999}]},{"osd":8,"last update":"Fri Jul 14 12:00:53 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.73699999999999999,"15min":0.70899999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.071,"5min":1.105,"15min":1.105},"last":1.117},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.70199999999999996,"15min":0.68999999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.85399999999999998}]},{"osd":9,"last update":"Fri Jul 14 12:00:18 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.81299999999999994,"15min":0.81100000000000005},"min":{"1min":0.60399999999999998,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.038,"5min":1.2,"15min":1.2},"last":1.149},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.77200000000000002,"15min":0.77500000000000002},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.026,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.96799999999999997}]},{"osd":10,"last update":"Fri Jul 14 12:00:38 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.71099999999999997,"15min":0.70699999999999996},"min":{"1min":0.55100000000000005,"5min":0.496,"15min":0.496},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.84899999999999998,"15min":0.85899999999999999},"min":{"1min":0.61599999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0580000000000001,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":0.83399999999999996}]},{"osd":11,"last update":"Fri Jul 14 12:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":1.1819999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.86699999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738446,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":1011712,"data_stored":488345,"data_compressed":17563,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 12:00:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.58899999999999997,"15min":0.55000000000000004},"min":{"1min":0.33600000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.96999999999999997,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.5},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.57699999999999996,"15min":0.53700000000000003},"min":{"1min":0.43099999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.92800000000000005,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.58999999999999997}]},{"osd":2,"last update":"Fri Jul 14 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.64400000000000002,"15min":0.59599999999999997},"min":{"1min":0.435,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.61599999999999999,"15min":0.56899999999999995},"min":{"1min":0.38200000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.89900000000000002,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.627}]},{"osd":3,"last update":"Fri Jul 14 11:58:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.54500000000000004,"15min":0.52300000000000002},"min":{"1min":0.313,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.55700000000000005,"15min":0.51100000000000001},"min":{"1min":0.39600000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.375}]},{"osd":4,"last update":"Fri Jul 14 12:00:23 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.623,"15min":0.58699999999999997},"min":{"1min":0.38500000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.64700000000000002,"15min":0.59399999999999997},"min":{"1min":0.45100000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.93200000000000005,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.60099999999999998}]},{"osd":5,"last update":"Fri Jul 14 12:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.68899999999999995,"15min":0.65600000000000003},"min":{"1min":0.45200000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.71599999999999997,"15min":0.68300000000000005},"min":{"1min":0.57299999999999995,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.96199999999999997,"5min":1.131,"15min":1.131},"last":0.78500000000000003}]},{"osd":6,"last update":"Fri Jul 14 12:00:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.67600000000000005,"15min":0.66000000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.92600000000000005,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.435},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.68700000000000006,"15min":0.68100000000000005},"min":{"1min":0.40999999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86899999999999999,"5min":1.081,"15min":1.081},"last":0.56499999999999995}]},{"osd":7,"last update":"Fri Jul 14 12:00:48 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.54400000000000004}]},{"osd":8,"last update":"Fri Jul 14 12:00:52 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.80700000000000005,"15min":0.80300000000000005},"min":{"1min":0.54200000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":3.089,"5min":3.089,"15min":3.089},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.876,"5min":0.82699999999999996,"15min":0.81999999999999995},"min":{"1min":0.67000000000000004,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.83299999999999996}]},{"osd":9,"last update":"Fri Jul 14 12:00:13 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.85799999999999998,"15min":0.86299999999999999},"min":{"1min":0.59699999999999998,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.747},{"interface":"front","average":{"1min":0.871,"5min":0.90700000000000003,"15min":0.91300000000000003},"min":{"1min":0.57899999999999996,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.204,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.79200000000000004}]},{"osd":10,"last update":"Fri Jul 14 12:00:35 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.92400000000000004,"15min":0.93200000000000005},"min":{"1min":0.66800000000000004,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.083,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.93100000000000005,"15min":0.94599999999999995},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.232,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.70999999999999996}]},{"osd":11,"last update":"Fri Jul 14 12:00:57 2023","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":0.86099999999999999,"15min":0.82699999999999996},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":4.4050000000000002,"5min":4.4050000000000002,"15min":4.4050000000000002},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.0720000000000001,"5min":0.90900000000000003,"15min":0.88200000000000001},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":4.3869999999999996,"5min":4.3869999999999996,"15min":4.3869999999999996},"last":0.80700000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":1015808,"data_stored":491885,"data_compressed":17563,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 12:00:40 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.56200000000000006,"15min":0.504},"min":{"1min":0.373,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.78200000000000003,"5min":6.54,"15min":6.54},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.54200000000000004,"15min":0.504},"min":{"1min":0.40600000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.42699999999999999}]},{"osd":1,"last update":"Fri Jul 14 12:00:40 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.56100000000000005,"15min":0.505},"min":{"1min":0.34000000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.89000000000000001,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.56699999999999995,"15min":0.51100000000000001},"min":{"1min":0.34799999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.86199999999999999,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.747}]},{"osd":3,"last update":"Fri Jul 14 12:00:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.58499999999999996,"15min":0.54600000000000004},"min":{"1min":0.39200000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.83299999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.59599999999999997,"15min":0.55900000000000005},"min":{"1min":0.41599999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.88400000000000001,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.68000000000000005}]},{"osd":4,"last update":"Fri Jul 14 12:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.73199999999999998}]},{"osd":5,"last update":"Fri Jul 14 12:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.47199999999999998,"15min":0.44400000000000001},"min":{"1min":0.28899999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.498,"15min":0.46100000000000002},"min":{"1min":0.318,"5min":0.252,"15min":0.252},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.57499999999999996}]},{"osd":6,"last update":"Fri Jul 14 12:00:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.65200000000000002,"15min":0.63700000000000001},"min":{"1min":0.50700000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.64800000000000002,"15min":0.63800000000000001},"min":{"1min":0.46600000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.47999999999999998}]},{"osd":7,"last update":"Fri Jul 14 12:00:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.84999999999999998}]},{"osd":8,"last update":"Fri Jul 14 12:00:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.61099999999999999,"15min":0.60399999999999998},"min":{"1min":0.35799999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.63900000000000001,"15min":0.627},"min":{"1min":0.41399999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.77600000000000002}]},{"osd":9,"last update":"Fri Jul 14 12:00:18 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.752,"15min":0.74199999999999999},"min":{"1min":0.46200000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.76500000000000001,"15min":0.753},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.82599999999999996}]},{"osd":10,"last update":"Fri Jul 14 12:00:40 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.76500000000000001,"15min":0.753},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.749,"15min":0.73599999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.86399999999999999}]},{"osd":11,"last update":"Fri Jul 14 12:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.877},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.81299999999999994}]}]},{"osd":3,"up_from":26,"seq":111669149761,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":999424,"data_stored":486645,"data_compressed":17563,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 12:00:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.498,"15min":0.40000000000000002},"min":{"1min":0.36299999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":2.7080000000000002,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.53600000000000003,"15min":0.42899999999999999},"min":{"1min":0.379,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":5.1790000000000003,"5min":5.1790000000000003,"15min":5.1790000000000003},"last":0.55600000000000005}]},{"osd":1,"last update":"Fri Jul 14 12:00:57 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.71499999999999997,"15min":0.59699999999999998},"min":{"1min":0.53200000000000003,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":2.7930000000000001,"5min":2.7930000000000001,"15min":2.7930000000000001},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.746,"5min":0.629,"15min":0.56100000000000005},"min":{"1min":0.55200000000000005,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.63700000000000001}]},{"osd":2,"last update":"Fri Jul 14 12:00:57 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.73699999999999999,"15min":0.628},"min":{"1min":0.59699999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":4.9610000000000003,"5min":4.9610000000000003,"15min":4.9610000000000003},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.73699999999999999,"15min":0.63400000000000001},"min":{"1min":0.53700000000000003,"5min":0.374,"15min":0.374},"max":{"1min":2.9249999999999998,"5min":2.9249999999999998,"15min":2.9249999999999998},"last":0.70899999999999996}]},{"osd":4,"last update":"Fri Jul 14 12:00:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.65800000000000003,"15min":0.61099999999999999},"min":{"1min":0.51200000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.871,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.622},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.72899999999999998,"15min":0.63700000000000001},"min":{"1min":0.56499999999999995,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":3.9100000000000001,"5min":3.9100000000000001,"15min":3.9100000000000001},"last":0.80800000000000005}]},{"osd":5,"last update":"Fri Jul 14 12:00:38 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.66900000000000004,"15min":0.63300000000000001},"min":{"1min":0.42499999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.77700000000000002,"15min":0.70299999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":5.1440000000000001,"5min":5.1440000000000001,"15min":5.1440000000000001},"last":0.73999999999999999}]},{"osd":6,"last update":"Fri Jul 14 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.749,"15min":0.71799999999999997},"min":{"1min":0.53300000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.97999999999999998,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":1.1890000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.69499999999999995,"15min":0.69399999999999995},"min":{"1min":0.44700000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.95699999999999996,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":1.036}]},{"osd":7,"last update":"Fri Jul 14 12:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.878,"15min":0.872},"min":{"1min":0.55500000000000005,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":3.931,"5min":3.931,"15min":3.931},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.89800000000000002,"15min":0.89000000000000001},"min":{"1min":0.66300000000000003,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.242,"5min":1.383,"15min":1.383},"last":1.107}]},{"osd":8,"last update":"Fri Jul 14 11:58:46 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":1.0009999999999999}]},{"osd":9,"last update":"Fri Jul 14 12:00:17 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":1.0069999999999999,"15min":1.0249999999999999},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.236,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":1.153},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.84899999999999998,"15min":0.86699999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0369999999999999,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.94799999999999995}]},{"osd":10,"last update":"Fri Jul 14 12:00:35 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.89100000000000001,"15min":0.88500000000000001},"min":{"1min":0.68200000000000005,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.863,"5min":1.863,"15min":1.863},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.0800000000000001,"5min":0.97099999999999997,"15min":0.95299999999999996},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":5.0140000000000002,"5min":5.0140000000000002,"15min":5.0140000000000002},"last":1.0509999999999999}]},{"osd":11,"last update":"Fri Jul 14 12:00:46 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.9980000000000002,"5min":2.9980000000000002,"15min":2.9980000000000002},"last":1.232},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.9379999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.1359999999999999}]}]},{"osd":4,"up_from":34,"seq":146028888125,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":991232,"data_stored":486867,"data_compressed":17563,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 11:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.52700000000000002,"15min":0.504},"min":{"1min":0.41399999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.53700000000000003,"15min":0.52300000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.80000000000000004,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.48299999999999998}]},{"osd":1,"last update":"Fri Jul 14 12:00:22 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.373,"15min":0.35499999999999998},"min":{"1min":0.29599999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.56899999999999995,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.38600000000000001,"15min":0.375},"min":{"1min":0.27000000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.55900000000000005,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.42199999999999999}]},{"osd":2,"last update":"Fri Jul 14 12:00:22 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.56100000000000005,"15min":0.54300000000000004},"min":{"1min":0.41099999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.59199999999999997,"15min":0.55700000000000005},"min":{"1min":0.432,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.83799999999999997,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.66500000000000004}]},{"osd":3,"last update":"Fri Jul 14 12:00:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57399999999999995,"15min":0.55200000000000005},"min":{"1min":0.35299999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.79600000000000004,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.58499999999999996,"15min":0.56699999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.63200000000000001}]},{"osd":5,"last update":"Fri Jul 14 12:00:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.60999999999999999,"15min":0.58499999999999996},"min":{"1min":0.45200000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.80600000000000005,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.60099999999999998,"15min":0.58899999999999997},"min":{"1min":0.39200000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.65200000000000002}]},{"osd":6,"last update":"Fri Jul 14 12:00:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.623,"15min":0.61599999999999999},"min":{"1min":0.46800000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.80800000000000005,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.69199999999999995,"15min":0.68400000000000005},"min":{"1min":0.48299999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.70699999999999996}]},{"osd":7,"last update":"Fri Jul 14 12:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56599999999999995,"15min":0.54700000000000004},"min":{"1min":0.34499999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.78300000000000003,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53900000000000003,"15min":0.51900000000000002},"min":{"1min":0.40600000000000003,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.71099999999999997,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.50700000000000001}]},{"osd":8,"last update":"Fri Jul 14 12:00:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.63500000000000001,"15min":0.621},"min":{"1min":0.44900000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.73399999999999999,"15min":0.72399999999999998},"min":{"1min":0.52000000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.94499999999999995,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.81899999999999995}]},{"osd":9,"last update":"Fri Jul 14 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71099999999999997,"15min":0.70999999999999996},"min":{"1min":0.55600000000000005,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.80000000000000004,"15min":0.80600000000000005},"min":{"1min":0.55900000000000005,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.92800000000000005,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.77200000000000002}]},{"osd":10,"last update":"Fri Jul 14 12:00:39 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.77100000000000002,"15min":0.77300000000000002},"min":{"1min":0.53900000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.91100000000000003,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.68899999999999995,"15min":0.69099999999999995},"min":{"1min":0.51800000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.79800000000000004}]},{"osd":11,"last update":"Fri Jul 14 12:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.87},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.83699999999999997}]}]},{"osd":5,"up_from":39,"seq":167503724601,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":1036,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":1060864,"data_stored":1079024,"data_compressed":53729,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 12:00:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.60199999999999998,"15min":0.60399999999999998},"min":{"1min":0.46600000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77500000000000002,"5min":1.018,"15min":1.018},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59799999999999998,"15min":0.60899999999999999},"min":{"1min":0.435,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.94399999999999995,"5min":1.127,"15min":1.127},"last":0.56299999999999994}]},{"osd":1,"last update":"Fri Jul 14 12:00:39 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.57799999999999996,"15min":0.53600000000000003},"min":{"1min":0.497,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.85599999999999998,"5min":1.018,"15min":1.018},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.58499999999999996,"15min":0.54500000000000004},"min":{"1min":0.48999999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.73799999999999999}]},{"osd":2,"last update":"Fri Jul 14 12:00:39 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.40600000000000003,"15min":0.38400000000000001},"min":{"1min":0.37,"5min":0.22,"15min":0.22},"max":{"1min":0.67100000000000004,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.42599999999999999,"15min":0.41299999999999998},"min":{"1min":0.39300000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.59499999999999997,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.68500000000000005}]},{"osd":3,"last update":"Fri Jul 14 12:00:39 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65700000000000003,"15min":0.64500000000000002},"min":{"1min":0.436,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.88800000000000001,"5min":3.6640000000000001,"15min":3.6640000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.66700000000000004,"15min":0.66000000000000003},"min":{"1min":0.52600000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.78900000000000003}]},{"osd":4,"last update":"Fri Jul 14 12:00:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.59199999999999997,"15min":0.56100000000000005},"min":{"1min":0.5,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.629},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.59799999999999998,"15min":0.57299999999999995},"min":{"1min":0.51000000000000001,"5min":0.379,"15min":0.379},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.64600000000000002}]},{"osd":6,"last update":"Fri Jul 14 12:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.69899999999999995,"15min":0.67800000000000005},"min":{"1min":0.44500000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.876},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.70299999999999996,"15min":0.69799999999999995},"min":{"1min":0.53300000000000003,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.97599999999999998,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.80400000000000005}]},{"osd":7,"last update":"Fri Jul 14 12:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.67300000000000004,"15min":0.66700000000000004},"min":{"1min":0.56200000000000006,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.73399999999999999,"15min":0.745},"min":{"1min":0.55600000000000005,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.095,"5min":3.6440000000000001,"15min":3.6440000000000001},"last":0.92300000000000004}]},{"osd":8,"last update":"Fri Jul 14 12:00:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.59799999999999998,"15min":0.57799999999999996},"min":{"1min":0.41699999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.879,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.58699999999999997,"15min":0.58099999999999996},"min":{"1min":0.442,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.70199999999999996}]},{"osd":9,"last update":"Fri Jul 14 12:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.79000000000000004,"15min":0.78400000000000003},"min":{"1min":0.59499999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.77600000000000002,"15min":0.77000000000000002},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.83699999999999997}]},{"osd":10,"last update":"Fri Jul 14 12:00:34 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.68799999999999994,"15min":0.67200000000000004},"min":{"1min":0.63,"5min":0.504,"15min":0.504},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.75600000000000001,"15min":0.74199999999999999},"min":{"1min":0.59199999999999997,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":1.038}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36166,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36166,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36166,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36166,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36166,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":5482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T12:01:09.018 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-07-14T12:01:09.019 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-07-14T12:01:09.019 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-07-14T12:01:09.019 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- ceph health --format=json 2023-07-14T12:01:09.084 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:08 smithi017 ceph-mon[126339]: pgmap v322: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-07-14T12:01:09.085 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:08 smithi017 ceph-mon[126339]: osdmap e92: 12 total, 12 up, 12 in 2023-07-14T12:01:09.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:08 smithi093 ceph-mon[128922]: pgmap v322: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-07-14T12:01:09.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:08 smithi093 ceph-mon[128922]: osdmap e92: 12 total, 12 up, 12 in 2023-07-14T12:01:09.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:08 smithi097 ceph-mon[129313]: pgmap v322: 129 pgs: 129 active+clean; 586 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-07-14T12:01:09.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:08 smithi097 ceph-mon[129313]: osdmap e92: 12 total, 12 up, 12 in 2023-07-14T12:01:09.753 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T12:01:10.051 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:09 smithi017 ceph-mon[126339]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T12:01:10.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:09 smithi093 ceph-mon[128922]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T12:01:10.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:09 smithi097 ceph-mon[129313]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T12:01:11.064 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:10 smithi017 ceph-mon[126339]: pgmap v324: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-07-14T12:01:11.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:10 smithi093 ceph-mon[128922]: pgmap v324: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-07-14T12:01:11.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:10 smithi097 ceph-mon[129313]: pgmap v324: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-07-14T12:01:11.337 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.356+0000 7f174293c700 1 -- 172.21.15.17:0/1274124933 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f173c0731c0 msgr2=0x7f173c10bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:11.337 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.356+0000 7f174293c700 1 --2- 172.21.15.17:0/1274124933 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f173c0731c0 0x7f173c10bea0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1730009990 tx=0x7f1730009ca0 comp rx=0 tx=0).stop 2023-07-14T12:01:11.337 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.357+0000 7f174293c700 1 -- 172.21.15.17:0/1274124933 shutdown_connections 2023-07-14T12:01:11.337 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.357+0000 7f174293c700 1 --2- 172.21.15.17:0/1274124933 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f173c10c3e0 0x7f173c10e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:11.338 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.357+0000 7f174293c700 1 --2- 172.21.15.17:0/1274124933 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f173c0728b0 0x7f173c072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:11.338 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.357+0000 7f174293c700 1 --2- 172.21.15.17:0/1274124933 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f173c0731c0 0x7f173c10bea0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:11.338 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.357+0000 7f174293c700 1 -- 172.21.15.17:0/1274124933 >> 172.21.15.17:0/1274124933 conn(0x7f173c0fb660 msgr2=0x7f173c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:11.338 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.357+0000 7f174293c700 1 -- 172.21.15.17:0/1274124933 shutdown_connections 2023-07-14T12:01:11.338 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.357+0000 7f174293c700 1 -- 172.21.15.17:0/1274124933 wait complete. 2023-07-14T12:01:11.340 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.358+0000 7f174293c700 1 Processor -- start 2023-07-14T12:01:11.340 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.358+0000 7f174293c700 1 -- start start 2023-07-14T12:01:11.340 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.358+0000 7f174293c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f173c0728b0 0x7f173c197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:11.340 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.358+0000 7f174293c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f173c0731c0 0x7f173c198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:11.341 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.358+0000 7f174293c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f173c10c3e0 0x7f173c19c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:11.341 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.358+0000 7f174293c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173c19ce40 con 0x7f173c0731c0 2023-07-14T12:01:11.341 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.358+0000 7f174293c700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173c19cfb0 con 0x7f173c10c3e0 2023-07-14T12:01:11.341 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.358+0000 7f174293c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173c19d170 con 0x7f173c0728b0 2023-07-14T12:01:11.341 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.359+0000 7f173b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f173c0731c0 0x7f173c198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:11.341 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.359+0000 7f173bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f173c0728b0 0x7f173c197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:11.341 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.359+0000 7f173bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f173c0728b0 0x7f173c197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.17:35744/0 (socket says 172.21.15.17:35744) 2023-07-14T12:01:11.342 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.359+0000 7f173b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f173c0731c0 0x7f173c198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49690/0 (socket says 172.21.15.17:49690) 2023-07-14T12:01:11.342 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.359+0000 7f173bfff700 1 -- 172.21.15.17:0/2158176999 learned_addr learned my addr 172.21.15.17:0/2158176999 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:11.342 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.359+0000 7f1740ed9700 1 --2- 172.21.15.17:0/2158176999 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f173c10c3e0 0x7f173c19c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:11.342 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.359+0000 7f173b7fe700 1 -- 172.21.15.17:0/2158176999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f173c0728b0 msgr2=0x7f173c197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:11.342 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.359+0000 7f173b7fe700 1 --2- 172.21.15.17:0/2158176999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f173c0728b0 0x7f173c197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:11.342 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.359+0000 7f173b7fe700 1 -- 172.21.15.17:0/2158176999 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f173c10c3e0 msgr2=0x7f173c19c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:11.342 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.359+0000 7f173b7fe700 1 --2- 172.21.15.17:0/2158176999 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f173c10c3e0 0x7f173c19c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:11.342 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.359+0000 7f173b7fe700 1 -- 172.21.15.17:0/2158176999 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17300095f0 con 0x7f173c0731c0 2023-07-14T12:01:11.343 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.359+0000 7f1740ed9700 1 --2- 172.21.15.17:0/2158176999 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f173c10c3e0 0x7f173c19c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T12:01:11.343 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.359+0000 7f173b7fe700 1 --2- 172.21.15.17:0/2158176999 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f173c0731c0 0x7f173c198330 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1730000c00 tx=0x7f1730005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:11.343 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.360+0000 7f17397fa700 1 -- 172.21.15.17:0/2158176999 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173001d070 con 0x7f173c0731c0 2023-07-14T12:01:11.343 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.360+0000 7f174293c700 1 -- 172.21.15.17:0/2158176999 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f173c19d3f0 con 0x7f173c0731c0 2023-07-14T12:01:11.343 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.360+0000 7f17397fa700 1 -- 172.21.15.17:0/2158176999 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1730021410 con 0x7f173c0731c0 2023-07-14T12:01:11.343 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.360+0000 7f17397fa700 1 -- 172.21.15.17:0/2158176999 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1730017650 con 0x7f173c0731c0 2023-07-14T12:01:11.343 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.360+0000 7f174293c700 1 -- 172.21.15.17:0/2158176999 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f173c1a4c20 con 0x7f173c0731c0 2023-07-14T12:01:11.344 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.361+0000 7f174293c700 1 -- 172.21.15.17:0/2158176999 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f173c04e210 con 0x7f173c0731c0 2023-07-14T12:01:11.344 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.363+0000 7f17397fa700 1 -- 172.21.15.17:0/2158176999 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f17300217f0 con 0x7f173c0731c0 2023-07-14T12:01:11.344 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.364+0000 7f17397fa700 1 --2- 172.21.15.17:0/2158176999 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f17240a1150 0x7f17240a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:11.345 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.364+0000 7f17397fa700 1 -- 172.21.15.17:0/2158176999 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f17300cdf40 con 0x7f173c0731c0 2023-07-14T12:01:11.345 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.365+0000 7f173bfff700 1 --2- 172.21.15.17:0/2158176999 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f17240a1150 0x7f17240a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:11.346 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.365+0000 7f173bfff700 1 --2- 172.21.15.17:0/2158176999 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f17240a1150 0x7f17240a3600 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f172c009d20 tx=0x7f172c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:11.348 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.368+0000 7f17397fa700 1 -- 172.21.15.17:0/2158176999 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1730097370 con 0x7f173c0731c0 2023-07-14T12:01:11.604 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.623+0000 7f174293c700 1 -- 172.21.15.17:0/2158176999 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f173c1a4db0 con 0x7f173c0731c0 2023-07-14T12:01:11.605 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.624+0000 7f17397fa700 1 -- 172.21.15.17:0/2158176999 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f173009a990 con 0x7f173c0731c0 2023-07-14T12:01:11.605 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T12:01:11.606 INFO:teuthology.orchestra.run.smithi017.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-14T12:01:11.607 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.626+0000 7f174293c700 1 -- 172.21.15.17:0/2158176999 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f17240a1150 msgr2=0x7f17240a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:11.607 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.626+0000 7f174293c700 1 --2- 172.21.15.17:0/2158176999 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f17240a1150 0x7f17240a3600 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f172c009d20 tx=0x7f172c009260 comp rx=0 tx=0).stop 2023-07-14T12:01:11.607 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.626+0000 7f174293c700 1 -- 172.21.15.17:0/2158176999 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f173c0731c0 msgr2=0x7f173c198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:11.608 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.626+0000 7f174293c700 1 --2- 172.21.15.17:0/2158176999 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f173c0731c0 0x7f173c198330 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1730000c00 tx=0x7f1730005740 comp rx=0 tx=0).stop 2023-07-14T12:01:11.608 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.626+0000 7f174293c700 1 -- 172.21.15.17:0/2158176999 shutdown_connections 2023-07-14T12:01:11.608 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.626+0000 7f174293c700 1 --2- 172.21.15.17:0/2158176999 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f17240a1150 0x7f17240a3600 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:11.608 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.626+0000 7f174293c700 1 --2- 172.21.15.17:0/2158176999 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f173c10c3e0 0x7f173c19c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:11.609 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.626+0000 7f174293c700 1 --2- 172.21.15.17:0/2158176999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f173c0728b0 0x7f173c197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:11.609 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.626+0000 7f174293c700 1 --2- 172.21.15.17:0/2158176999 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f173c0731c0 0x7f173c198330 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:11.609 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.626+0000 7f174293c700 1 -- 172.21.15.17:0/2158176999 >> 172.21.15.17:0/2158176999 conn(0x7f173c0fb660 msgr2=0x7f173c10e490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:11.609 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.626+0000 7f174293c700 1 -- 172.21.15.17:0/2158176999 shutdown_connections 2023-07-14T12:01:11.610 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:11.627+0000 7f174293c700 1 -- 172.21.15.17:0/2158176999 wait complete. 2023-07-14T12:01:11.893 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:11 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/2158176999' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-07-14T12:01:12.055 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-07-14T12:01:12.056 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T12:01:12.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:11 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/2158176999' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-07-14T12:01:12.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:11 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/2158176999' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-07-14T12:01:12.266 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.286+0000 7efc423ff700 1 -- 172.21.15.17:0/2063421421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc3c0ffba0 msgr2=0x7efc3c0fffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:12.266 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.286+0000 7efc423ff700 1 --2- 172.21.15.17:0/2063421421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc3c0ffba0 0x7efc3c0fffb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7efc2c009990 tx=0x7efc2c009ca0 comp rx=0 tx=0).stop 2023-07-14T12:01:12.266 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.286+0000 7efc423ff700 1 -- 172.21.15.17:0/2063421421 shutdown_connections 2023-07-14T12:01:12.266 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.286+0000 7efc423ff700 1 --2- 172.21.15.17:0/2063421421 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efc3c1017e0 0x7efc3c108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:12.267 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.286+0000 7efc423ff700 1 --2- 172.21.15.17:0/2063421421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc3c0ffba0 0x7efc3c0fffb0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:12.267 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.286+0000 7efc423ff700 1 --2- 172.21.15.17:0/2063421421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc3c100e30 0x7efc3c1012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:12.267 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.286+0000 7efc423ff700 1 -- 172.21.15.17:0/2063421421 >> 172.21.15.17:0/2063421421 conn(0x7efc3c0fb1c0 msgr2=0x7efc3c0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:12.267 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.286+0000 7efc423ff700 1 -- 172.21.15.17:0/2063421421 shutdown_connections 2023-07-14T12:01:12.267 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.286+0000 7efc423ff700 1 -- 172.21.15.17:0/2063421421 wait complete. 2023-07-14T12:01:12.268 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.287+0000 7efc423ff700 1 Processor -- start 2023-07-14T12:01:12.269 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.287+0000 7efc423ff700 1 -- start start 2023-07-14T12:01:12.269 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.287+0000 7efc423ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc3c0ffba0 0x7efc3c1958f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:12.269 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.287+0000 7efc423ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc3c100e30 0x7efc3c195e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:12.269 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.287+0000 7efc423ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efc3c1017e0 0x7efc3c19cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:12.269 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.287+0000 7efc423ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c19d470 con 0x7efc3c0ffba0 2023-07-14T12:01:12.269 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.287+0000 7efc423ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c19d5b0 con 0x7efc3c1017e0 2023-07-14T12:01:12.270 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.287+0000 7efc423ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c19d740 con 0x7efc3c100e30 2023-07-14T12:01:12.270 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.288+0000 7efc3bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc3c0ffba0 0x7efc3c1958f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:12.270 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.288+0000 7efc3bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc3c0ffba0 0x7efc3c1958f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49708/0 (socket says 172.21.15.17:49708) 2023-07-14T12:01:12.270 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.288+0000 7efc3bfff700 1 -- 172.21.15.17:0/2373412616 learned_addr learned my addr 172.21.15.17:0/2373412616 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:12.270 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.288+0000 7efc4099c700 1 --2- 172.21.15.17:0/2373412616 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efc3c1017e0 0x7efc3c19cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:12.270 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.288+0000 7efc3b7fe700 1 --2- 172.21.15.17:0/2373412616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc3c100e30 0x7efc3c195e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:12.270 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.288+0000 7efc3bfff700 1 -- 172.21.15.17:0/2373412616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc3c100e30 msgr2=0x7efc3c195e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:12.270 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.288+0000 7efc3bfff700 1 --2- 172.21.15.17:0/2373412616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc3c100e30 0x7efc3c195e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:12.271 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.288+0000 7efc3bfff700 1 -- 172.21.15.17:0/2373412616 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efc3c1017e0 msgr2=0x7efc3c19cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:12.271 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.288+0000 7efc3bfff700 1 --2- 172.21.15.17:0/2373412616 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efc3c1017e0 0x7efc3c19cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:12.271 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.288+0000 7efc3bfff700 1 -- 172.21.15.17:0/2373412616 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc2c0095f0 con 0x7efc3c0ffba0 2023-07-14T12:01:12.271 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.288+0000 7efc3bfff700 1 --2- 172.21.15.17:0/2373412616 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc3c0ffba0 0x7efc3c1958f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efc2c004740 tx=0x7efc2c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:12.271 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.289+0000 7efc397fa700 1 -- 172.21.15.17:0/2373412616 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2c01d070 con 0x7efc3c0ffba0 2023-07-14T12:01:12.271 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.289+0000 7efc423ff700 1 -- 172.21.15.17:0/2373412616 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc3c19d9c0 con 0x7efc3c0ffba0 2023-07-14T12:01:12.271 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.289+0000 7efc397fa700 1 -- 172.21.15.17:0/2373412616 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc2c004830 con 0x7efc3c0ffba0 2023-07-14T12:01:12.271 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.289+0000 7efc397fa700 1 -- 172.21.15.17:0/2373412616 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2c0053c0 con 0x7efc3c0ffba0 2023-07-14T12:01:12.272 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.289+0000 7efc423ff700 1 -- 172.21.15.17:0/2373412616 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc3c19dde0 con 0x7efc3c0ffba0 2023-07-14T12:01:12.273 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.290+0000 7efc423ff700 1 -- 172.21.15.17:0/2373412616 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc3c106b70 con 0x7efc3c0ffba0 2023-07-14T12:01:12.273 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.292+0000 7efc397fa700 1 -- 172.21.15.17:0/2373412616 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7efc2c005520 con 0x7efc3c0ffba0 2023-07-14T12:01:12.273 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.293+0000 7efc397fa700 1 --2- 172.21.15.17:0/2373412616 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7efc240a1150 0x7efc240a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:12.274 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.294+0000 7efc397fa700 1 -- 172.21.15.17:0/2373412616 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7efc2c0ce1c0 con 0x7efc3c0ffba0 2023-07-14T12:01:12.274 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.294+0000 7efc3b7fe700 1 --2- 172.21.15.17:0/2373412616 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7efc240a1150 0x7efc240a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:12.274 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.294+0000 7efc3b7fe700 1 --2- 172.21.15.17:0/2373412616 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7efc240a1150 0x7efc240a3600 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efc30009c60 tx=0x7efc30009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:12.277 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.297+0000 7efc397fa700 1 -- 172.21.15.17:0/2373412616 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7efc2c0975f0 con 0x7efc3c0ffba0 2023-07-14T12:01:12.486 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.506+0000 7efc423ff700 1 -- 172.21.15.17:0/2373412616 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efc3c02c850 con 0x7efc3c0ffba0 2023-07-14T12:01:12.487 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.507+0000 7efc397fa700 1 -- 172.21.15.17:0/2373412616 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24486 (secure 0 0 0) 0x7efc2c09ac10 con 0x7efc3c0ffba0 2023-07-14T12:01:12.488 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T12:01:12.488 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":92,"fsid":"b3c386cc-223c-11ee-9b33-001a4aab830c","created":"2023-07-14T11:51:59.992960+0000","modified":"2023-07-14T12:01:07.806062+0000","last_up_change":"2023-07-14T11:58:55.037769+0000","last_in_change":"2023-07-14T11:58:37.114870+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:55:28.237002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:59:41.560502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T11:59:42.725461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:59:46.267029+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"57003438-a83c-4c8e-a029-56414740895c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6803","nonce":3588366384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6805","nonce":3588366384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6809","nonce":3588366384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6807","nonce":3588366384}]},"public_addr":"172.21.15.17:6803/3588366384","cluster_addr":"172.21.15.17:6805/3588366384","heartbeat_back_addr":"172.21.15.17:6809/3588366384","heartbeat_front_addr":"172.21.15.17:6807/3588366384","state":["exists","up"]},{"osd":1,"uuid":"f56b6337-d8a9-4411-b9d8-b6863dc98556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6801","nonce":3147203064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6803","nonce":3147203064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6807","nonce":3147203064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6805","nonce":3147203064}]},"public_addr":"172.21.15.93:6801/3147203064","cluster_addr":"172.21.15.93:6803/3147203064","heartbeat_back_addr":"172.21.15.93:6807/3147203064","heartbeat_front_addr":"172.21.15.93:6805/3147203064","state":["exists","up"]},{"osd":2,"uuid":"fc34bb7f-551f-46e1-99c0-59924d6c1346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6801","nonce":1877269815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6803","nonce":1877269815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6807","nonce":1877269815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6805","nonce":1877269815}]},"public_addr":"172.21.15.97:6801/1877269815","cluster_addr":"172.21.15.97:6803/1877269815","heartbeat_back_addr":"172.21.15.97:6807/1877269815","heartbeat_front_addr":"172.21.15.97:6805/1877269815","state":["exists","up"]},{"osd":3,"uuid":"37a1583c-4fb8-412a-a0c8-f83b1ff26a91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6811","nonce":2784619821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6813","nonce":2784619821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6817","nonce":2784619821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6815","nonce":2784619821}]},"public_addr":"172.21.15.17:6811/2784619821","cluster_addr":"172.21.15.17:6813/2784619821","heartbeat_back_addr":"172.21.15.17:6817/2784619821","heartbeat_front_addr":"172.21.15.17:6815/2784619821","state":["exists","up"]},{"osd":4,"uuid":"f4b9f377-d519-4b14-ac4a-f1debaeed9f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6809","nonce":3151801173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6811","nonce":3151801173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6815","nonce":3151801173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6813","nonce":3151801173}]},"public_addr":"172.21.15.93:6809/3151801173","cluster_addr":"172.21.15.93:6811/3151801173","heartbeat_back_addr":"172.21.15.93:6815/3151801173","heartbeat_front_addr":"172.21.15.93:6813/3151801173","state":["exists","up"]},{"osd":5,"uuid":"2d2af354-0535-4ef9-b166-3ef219498db8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6809","nonce":2184413924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6811","nonce":2184413924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6815","nonce":2184413924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6813","nonce":2184413924}]},"public_addr":"172.21.15.97:6809/2184413924","cluster_addr":"172.21.15.97:6811/2184413924","heartbeat_back_addr":"172.21.15.97:6815/2184413924","heartbeat_front_addr":"172.21.15.97:6813/2184413924","state":["exists","up"]},{"osd":6,"uuid":"9cdf0c4b-62dc-4168-864e-a9d1650a9ea8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6819","nonce":50486581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6821","nonce":50486581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6825","nonce":50486581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6823","nonce":50486581}]},"public_addr":"172.21.15.17:6819/50486581","cluster_addr":"172.21.15.17:6821/50486581","heartbeat_back_addr":"172.21.15.17:6825/50486581","heartbeat_front_addr":"172.21.15.17:6823/50486581","state":["exists","up"]},{"osd":7,"uuid":"09f60823-5491-45cd-9324-6ae0d56ce262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6817","nonce":2596912392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6819","nonce":2596912392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6823","nonce":2596912392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6821","nonce":2596912392}]},"public_addr":"172.21.15.93:6817/2596912392","cluster_addr":"172.21.15.93:6819/2596912392","heartbeat_back_addr":"172.21.15.93:6823/2596912392","heartbeat_front_addr":"172.21.15.93:6821/2596912392","state":["exists","up"]},{"osd":8,"uuid":"8c5771c9-718e-4313-a2b6-5e1c37c53f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6817","nonce":1965492859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6819","nonce":1965492859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6823","nonce":1965492859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6821","nonce":1965492859}]},"public_addr":"172.21.15.97:6817/1965492859","cluster_addr":"172.21.15.97:6819/1965492859","heartbeat_back_addr":"172.21.15.97:6823/1965492859","heartbeat_front_addr":"172.21.15.97:6821/1965492859","state":["exists","up"]},{"osd":9,"uuid":"1197b8a3-c09f-4c57-aba1-9e988a1620b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6827","nonce":2512645700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6829","nonce":2512645700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6833","nonce":2512645700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6831","nonce":2512645700}]},"public_addr":"172.21.15.17:6827/2512645700","cluster_addr":"172.21.15.17:6829/2512645700","heartbeat_back_addr":"172.21.15.17:6833/2512645700","heartbeat_front_addr":"172.21.15.17:6831/2512645700","state":["exists","up"]},{"osd":10,"uuid":"f99918c2-cdd8-4a7a-be2e-d1d6e0f828a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6825","nonce":264429558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6827","nonce":264429558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6831","nonce":264429558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6829","nonce":264429558}]},"public_addr":"172.21.15.93:6825/264429558","cluster_addr":"172.21.15.93:6827/264429558","heartbeat_back_addr":"172.21.15.93:6831/264429558","heartbeat_front_addr":"172.21.15.93:6829/264429558","state":["exists","up"]},{"osd":11,"uuid":"6bbb5b79-8d4d-441f-9583-203df6c78cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6825","nonce":3389913754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6827","nonce":3389913754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6831","nonce":3389913754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6829","nonce":3389913754}]},"public_addr":"172.21.15.97:6825/3389913754","cluster_addr":"172.21.15.97:6827/3389913754","heartbeat_back_addr":"172.21.15.97:6831/3389913754","heartbeat_front_addr":"172.21.15.97:6829/3389913754","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:54:41.079618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:03.207899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:24.136961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:50.392609+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:56:12.136237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:56:33.212314+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:00.243646+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:22.059234+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:43.232399+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:10.043896+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:31.910780+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:53.158983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.9","mappings":[{"from":5,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/2084033276":"2023-07-15T11:53:05.639597+0000","172.21.15.17:6801/1059284716":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/3651785636":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/1373538024":"2023-07-15T11:52:37.887946+0000","172.21.15.17:0/3560395020":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6800/1059284716":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/4262912861":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/2258375202":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6800/2804227742":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6801/2804227742":"2023-07-15T11:52:37.887946+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T12:01:12.491 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.510+0000 7efc423ff700 1 -- 172.21.15.17:0/2373412616 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7efc240a1150 msgr2=0x7efc240a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:12.491 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.510+0000 7efc423ff700 1 --2- 172.21.15.17:0/2373412616 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7efc240a1150 0x7efc240a3600 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efc30009c60 tx=0x7efc30009260 comp rx=0 tx=0).stop 2023-07-14T12:01:12.491 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.510+0000 7efc423ff700 1 -- 172.21.15.17:0/2373412616 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc3c0ffba0 msgr2=0x7efc3c1958f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:12.491 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.510+0000 7efc423ff700 1 --2- 172.21.15.17:0/2373412616 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc3c0ffba0 0x7efc3c1958f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efc2c004740 tx=0x7efc2c009580 comp rx=0 tx=0).stop 2023-07-14T12:01:12.492 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.510+0000 7efc423ff700 1 -- 172.21.15.17:0/2373412616 shutdown_connections 2023-07-14T12:01:12.492 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.510+0000 7efc423ff700 1 --2- 172.21.15.17:0/2373412616 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7efc240a1150 0x7efc240a3600 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:12.492 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.510+0000 7efc423ff700 1 --2- 172.21.15.17:0/2373412616 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efc3c1017e0 0x7efc3c19cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:12.493 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.510+0000 7efc423ff700 1 --2- 172.21.15.17:0/2373412616 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc3c0ffba0 0x7efc3c1958f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:12.493 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.510+0000 7efc423ff700 1 --2- 172.21.15.17:0/2373412616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc3c100e30 0x7efc3c195e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:12.493 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.510+0000 7efc423ff700 1 -- 172.21.15.17:0/2373412616 >> 172.21.15.17:0/2373412616 conn(0x7efc3c0fb1c0 msgr2=0x7efc3c0fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:12.493 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.510+0000 7efc423ff700 1 -- 172.21.15.17:0/2373412616 shutdown_connections 2023-07-14T12:01:12.494 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.510+0000 7efc423ff700 1 -- 172.21.15.17:0/2373412616 wait complete. 2023-07-14T12:01:12.510 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:55:28.237002+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T11:59:41.560502+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T11:59:42.725461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-14T11:59:46.267029+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-14T12:01:12.511 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T12:01:12.719 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.738+0000 7fce48fff700 1 -- 172.21.15.17:0/3728694429 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce441006e0 msgr2=0x7fce44102ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:12.719 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.738+0000 7fce48fff700 1 --2- 172.21.15.17:0/3728694429 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce441006e0 0x7fce44102ac0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fce34009990 tx=0x7fce34009ca0 comp rx=0 tx=0).stop 2023-07-14T12:01:12.719 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.738+0000 7fce48fff700 1 -- 172.21.15.17:0/3728694429 shutdown_connections 2023-07-14T12:01:12.720 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.738+0000 7fce48fff700 1 --2- 172.21.15.17:0/3728694429 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce44105920 0x7fce44107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:12.720 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.738+0000 7fce48fff700 1 --2- 172.21.15.17:0/3728694429 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce441006e0 0x7fce44102ac0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:12.720 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.738+0000 7fce48fff700 1 --2- 172.21.15.17:0/3728694429 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce44103000 0x7fce441053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:12.720 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.738+0000 7fce48fff700 1 -- 172.21.15.17:0/3728694429 >> 172.21.15.17:0/3728694429 conn(0x7fce440fa120 msgr2=0x7fce440fc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:12.720 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.738+0000 7fce48fff700 1 -- 172.21.15.17:0/3728694429 shutdown_connections 2023-07-14T12:01:12.721 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.738+0000 7fce48fff700 1 -- 172.21.15.17:0/3728694429 wait complete. 2023-07-14T12:01:12.722 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.739+0000 7fce48fff700 1 Processor -- start 2023-07-14T12:01:12.723 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.739+0000 7fce48fff700 1 -- start start 2023-07-14T12:01:12.723 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.740+0000 7fce48fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce441006e0 0x7fce44100050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:12.723 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.740+0000 7fce48fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce44103000 0x7fce440fe690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:12.723 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.740+0000 7fce48fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce44105920 0x7fce440febd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:12.723 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.740+0000 7fce48fff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce440ff370 con 0x7fce44103000 2023-07-14T12:01:12.723 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.740+0000 7fce48fff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce440ff4e0 con 0x7fce44105920 2023-07-14T12:01:12.723 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.740+0000 7fce48fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4419f940 con 0x7fce441006e0 2023-07-14T12:01:12.724 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.740+0000 7fce3bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce44103000 0x7fce440fe690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:12.724 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.740+0000 7fce3bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce44103000 0x7fce440fe690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49720/0 (socket says 172.21.15.17:49720) 2023-07-14T12:01:12.724 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.740+0000 7fce3bfff700 1 -- 172.21.15.17:0/2891646589 learned_addr learned my addr 172.21.15.17:0/2891646589 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:12.724 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.740+0000 7fce4259c700 1 --2- 172.21.15.17:0/2891646589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce441006e0 0x7fce44100050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:12.724 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.740+0000 7fce42d9d700 1 --2- 172.21.15.17:0/2891646589 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce44105920 0x7fce440febd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:12.724 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.740+0000 7fce3bfff700 1 -- 172.21.15.17:0/2891646589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce441006e0 msgr2=0x7fce44100050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:12.724 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.740+0000 7fce3bfff700 1 --2- 172.21.15.17:0/2891646589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce441006e0 0x7fce44100050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:12.725 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.740+0000 7fce3bfff700 1 -- 172.21.15.17:0/2891646589 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce44105920 msgr2=0x7fce440febd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:12.725 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.740+0000 7fce3bfff700 1 --2- 172.21.15.17:0/2891646589 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce44105920 0x7fce440febd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:12.725 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.740+0000 7fce3bfff700 1 -- 172.21.15.17:0/2891646589 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce340095f0 con 0x7fce44103000 2023-07-14T12:01:12.725 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.741+0000 7fce3bfff700 1 --2- 172.21.15.17:0/2891646589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce44103000 0x7fce440fe690 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fce2c00e9a0 tx=0x7fce2c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:12.725 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.741+0000 7fce3b7fe700 1 -- 172.21.15.17:0/2891646589 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c009790 con 0x7fce44103000 2023-07-14T12:01:12.725 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.741+0000 7fce3b7fe700 1 -- 172.21.15.17:0/2891646589 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce2c010430 con 0x7fce44103000 2023-07-14T12:01:12.725 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.741+0000 7fce3b7fe700 1 -- 172.21.15.17:0/2891646589 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c018680 con 0x7fce44103000 2023-07-14T12:01:12.725 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.741+0000 7fce48fff700 1 -- 172.21.15.17:0/2891646589 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce4419fc20 con 0x7fce44103000 2023-07-14T12:01:12.726 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.741+0000 7fce48fff700 1 -- 172.21.15.17:0/2891646589 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce441a00e0 con 0x7fce44103000 2023-07-14T12:01:12.726 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.742+0000 7fce48fff700 1 -- 172.21.15.17:0/2891646589 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce440fcca0 con 0x7fce44103000 2023-07-14T12:01:12.726 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.744+0000 7fce3b7fe700 1 -- 172.21.15.17:0/2891646589 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fce2c0105a0 con 0x7fce44103000 2023-07-14T12:01:12.727 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.746+0000 7fce3b7fe700 1 --2- 172.21.15.17:0/2891646589 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fce300a11d0 0x7fce300a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:12.727 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.746+0000 7fce3b7fe700 1 -- 172.21.15.17:0/2891646589 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fce2c014070 con 0x7fce44103000 2023-07-14T12:01:12.727 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.746+0000 7fce4259c700 1 --2- 172.21.15.17:0/2891646589 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fce300a11d0 0x7fce300a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:12.727 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.747+0000 7fce4259c700 1 --2- 172.21.15.17:0/2891646589 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fce300a11d0 0x7fce300a3680 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fce3400b2e0 tx=0x7fce34005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:12.730 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.750+0000 7fce3b7fe700 1 -- 172.21.15.17:0/2891646589 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fce2c051790 con 0x7fce44103000 2023-07-14T12:01:12.940 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.960+0000 7fce48fff700 1 -- 172.21.15.17:0/2891646589 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fce440611c0 con 0x7fce44103000 2023-07-14T12:01:12.941 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.960+0000 7fce3b7fe700 1 -- 172.21.15.17:0/2891646589 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7fce2c0961b0 con 0x7fce44103000 2023-07-14T12:01:12.941 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2023-07-14T12:01:12.943 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.963+0000 7fce48fff700 1 -- 172.21.15.17:0/2891646589 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fce300a11d0 msgr2=0x7fce300a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:12.943 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.963+0000 7fce48fff700 1 --2- 172.21.15.17:0/2891646589 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fce300a11d0 0x7fce300a3680 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fce3400b2e0 tx=0x7fce34005fb0 comp rx=0 tx=0).stop 2023-07-14T12:01:12.944 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.963+0000 7fce48fff700 1 -- 172.21.15.17:0/2891646589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce44103000 msgr2=0x7fce440fe690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:12.944 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.963+0000 7fce48fff700 1 --2- 172.21.15.17:0/2891646589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce44103000 0x7fce440fe690 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fce2c00e9a0 tx=0x7fce2c00ecb0 comp rx=0 tx=0).stop 2023-07-14T12:01:12.944 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.963+0000 7fce48fff700 1 -- 172.21.15.17:0/2891646589 shutdown_connections 2023-07-14T12:01:12.944 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.963+0000 7fce48fff700 1 --2- 172.21.15.17:0/2891646589 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fce300a11d0 0x7fce300a3680 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:12.944 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.963+0000 7fce48fff700 1 --2- 172.21.15.17:0/2891646589 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce44105920 0x7fce440febd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:12.944 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.963+0000 7fce48fff700 1 --2- 172.21.15.17:0/2891646589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce441006e0 0x7fce44100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:12.944 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.963+0000 7fce48fff700 1 --2- 172.21.15.17:0/2891646589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce44103000 0x7fce440fe690 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:12.945 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.963+0000 7fce48fff700 1 -- 172.21.15.17:0/2891646589 >> 172.21.15.17:0/2891646589 conn(0x7fce440fa120 msgr2=0x7fce44101f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:12.945 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.963+0000 7fce48fff700 1 -- 172.21.15.17:0/2891646589 shutdown_connections 2023-07-14T12:01:12.945 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:12.963+0000 7fce48fff700 1 -- 172.21.15.17:0/2891646589 wait complete. 2023-07-14T12:01:12.962 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T12:01:13.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:12 smithi093 ceph-mon[128922]: pgmap v325: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 7 op/s; 13 B/s, 0 objects/s recovering 2023-07-14T12:01:13.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:12 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/2373412616' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T12:01:13.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:12 smithi097 ceph-mon[129313]: pgmap v325: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 7 op/s; 13 B/s, 0 objects/s recovering 2023-07-14T12:01:13.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:12 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/2373412616' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T12:01:13.143 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:12 smithi017 ceph-mon[126339]: pgmap v325: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 7 op/s; 13 B/s, 0 objects/s recovering 2023-07-14T12:01:13.143 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:12 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/2373412616' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T12:01:13.170 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.190+0000 7f940c9ff700 1 -- 172.21.15.17:0/1465421464 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94080ffbe0 msgr2=0x7f94080ffff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:13.170 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.190+0000 7f940c9ff700 1 --2- 172.21.15.17:0/1465421464 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94080ffbe0 0x7f94080ffff0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f93f8009990 tx=0x7f93f8009ca0 comp rx=0 tx=0).stop 2023-07-14T12:01:13.170 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.190+0000 7f940c9ff700 1 -- 172.21.15.17:0/1465421464 shutdown_connections 2023-07-14T12:01:13.171 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.190+0000 7f940c9ff700 1 --2- 172.21.15.17:0/1465421464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9408101760 0x7f9408108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:13.171 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.190+0000 7f940c9ff700 1 --2- 172.21.15.17:0/1465421464 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94080ffbe0 0x7f94080ffff0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:13.171 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.190+0000 7f940c9ff700 1 --2- 172.21.15.17:0/1465421464 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9408100dd0 0x7f9408101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:13.171 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.190+0000 7f940c9ff700 1 -- 172.21.15.17:0/1465421464 >> 172.21.15.17:0/1465421464 conn(0x7f94080fb200 msgr2=0x7f94080fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:13.171 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.190+0000 7f940c9ff700 1 -- 172.21.15.17:0/1465421464 shutdown_connections 2023-07-14T12:01:13.171 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.190+0000 7f940c9ff700 1 -- 172.21.15.17:0/1465421464 wait complete. 2023-07-14T12:01:13.173 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.191+0000 7f940c9ff700 1 Processor -- start 2023-07-14T12:01:13.173 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.191+0000 7f940c9ff700 1 -- start start 2023-07-14T12:01:13.173 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.191+0000 7f940c9ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94080ffbe0 0x7f9408197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:13.174 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.191+0000 7f940c9ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9408100dd0 0x7f9408198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:13.174 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.191+0000 7f940c9ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9408101760 0x7f940819f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:13.174 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.191+0000 7f940c9ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f940819f750 con 0x7f94080ffbe0 2023-07-14T12:01:13.174 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.191+0000 7f940c9ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f940819f890 con 0x7f9408100dd0 2023-07-14T12:01:13.174 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.191+0000 7f940c9ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f940819fa20 con 0x7f9408101760 2023-07-14T12:01:13.174 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.192+0000 7f940659c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94080ffbe0 0x7f9408197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:13.174 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.192+0000 7f940659c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94080ffbe0 0x7f9408197bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53394/0 (socket says 172.21.15.17:53394) 2023-07-14T12:01:13.175 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.192+0000 7f940659c700 1 -- 172.21.15.17:0/4034612346 learned_addr learned my addr 172.21.15.17:0/4034612346 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:13.175 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.192+0000 7f9406d9d700 1 --2- 172.21.15.17:0/4034612346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9408101760 0x7f940819f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:13.175 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.192+0000 7f9405d9b700 1 --2- 172.21.15.17:0/4034612346 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9408100dd0 0x7f9408198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:13.175 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.192+0000 7f940659c700 1 -- 172.21.15.17:0/4034612346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9408101760 msgr2=0x7f940819f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:13.175 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.192+0000 7f940659c700 1 --2- 172.21.15.17:0/4034612346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9408101760 0x7f940819f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:13.176 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.192+0000 7f940659c700 1 -- 172.21.15.17:0/4034612346 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9408100dd0 msgr2=0x7f9408198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:13.176 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.192+0000 7f940659c700 1 --2- 172.21.15.17:0/4034612346 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9408100dd0 0x7f9408198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:13.176 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.192+0000 7f940659c700 1 -- 172.21.15.17:0/4034612346 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f80095f0 con 0x7f94080ffbe0 2023-07-14T12:01:13.176 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.192+0000 7f940659c700 1 --2- 172.21.15.17:0/4034612346 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94080ffbe0 0x7f9408197bd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f93f8004730 tx=0x7f93f8004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:13.176 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.193+0000 7f93f77fe700 1 -- 172.21.15.17:0/4034612346 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f801d070 con 0x7f94080ffbe0 2023-07-14T12:01:13.176 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.193+0000 7f93f77fe700 1 -- 172.21.15.17:0/4034612346 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93f8021410 con 0x7f94080ffbe0 2023-07-14T12:01:13.176 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.193+0000 7f940c9ff700 1 -- 172.21.15.17:0/4034612346 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f940819fca0 con 0x7f94080ffbe0 2023-07-14T12:01:13.177 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.193+0000 7f93f77fe700 1 -- 172.21.15.17:0/4034612346 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f8017650 con 0x7f94080ffbe0 2023-07-14T12:01:13.177 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.193+0000 7f940c9ff700 1 -- 172.21.15.17:0/4034612346 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94081a00c0 con 0x7f94080ffbe0 2023-07-14T12:01:13.177 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.194+0000 7f940c9ff700 1 -- 172.21.15.17:0/4034612346 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94081922e0 con 0x7f94080ffbe0 2023-07-14T12:01:13.178 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.196+0000 7f93f77fe700 1 -- 172.21.15.17:0/4034612346 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f93f8021580 con 0x7f94080ffbe0 2023-07-14T12:01:13.178 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.197+0000 7f93f77fe700 1 --2- 172.21.15.17:0/4034612346 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f93f00a1150 0x7f93f00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:13.178 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.197+0000 7f93f77fe700 1 -- 172.21.15.17:0/4034612346 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f93f80ce0a0 con 0x7f94080ffbe0 2023-07-14T12:01:13.178 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.197+0000 7f9405d9b700 1 --2- 172.21.15.17:0/4034612346 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f93f00a1150 0x7f93f00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:13.178 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.198+0000 7f9405d9b700 1 --2- 172.21.15.17:0/4034612346 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f93f00a1150 0x7f93f00a3600 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f93fc009c60 tx=0x7f93fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:13.181 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.201+0000 7f93f77fe700 1 -- 172.21.15.17:0/4034612346 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f93f80974d0 con 0x7f94080ffbe0 2023-07-14T12:01:13.388 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.408+0000 7f940c9ff700 1 -- 172.21.15.17:0/4034612346 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9408061120 con 0x7f94080ffbe0 2023-07-14T12:01:13.389 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.409+0000 7f93f77fe700 1 -- 172.21.15.17:0/4034612346 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f93f809aaf0 con 0x7f94080ffbe0 2023-07-14T12:01:13.390 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2023-07-14T12:01:13.392 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.411+0000 7f940c9ff700 1 -- 172.21.15.17:0/4034612346 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f93f00a1150 msgr2=0x7f93f00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:13.392 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.411+0000 7f940c9ff700 1 --2- 172.21.15.17:0/4034612346 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f93f00a1150 0x7f93f00a3600 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f93fc009c60 tx=0x7f93fc009260 comp rx=0 tx=0).stop 2023-07-14T12:01:13.393 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.411+0000 7f940c9ff700 1 -- 172.21.15.17:0/4034612346 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94080ffbe0 msgr2=0x7f9408197bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:13.393 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.411+0000 7f940c9ff700 1 --2- 172.21.15.17:0/4034612346 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94080ffbe0 0x7f9408197bd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f93f8004730 tx=0x7f93f8004760 comp rx=0 tx=0).stop 2023-07-14T12:01:13.393 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.411+0000 7f940c9ff700 1 -- 172.21.15.17:0/4034612346 shutdown_connections 2023-07-14T12:01:13.393 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.411+0000 7f940c9ff700 1 --2- 172.21.15.17:0/4034612346 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f93f00a1150 0x7f93f00a3600 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:13.393 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.411+0000 7f940c9ff700 1 --2- 172.21.15.17:0/4034612346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9408101760 0x7f940819f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:13.394 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.411+0000 7f940c9ff700 1 --2- 172.21.15.17:0/4034612346 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94080ffbe0 0x7f9408197bd0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:13.394 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.411+0000 7f940c9ff700 1 --2- 172.21.15.17:0/4034612346 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9408100dd0 0x7f9408198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:13.394 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.412+0000 7f940c9ff700 1 -- 172.21.15.17:0/4034612346 >> 172.21.15.17:0/4034612346 conn(0x7f94080fb200 msgr2=0x7f94081066b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:13.394 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.412+0000 7f940c9ff700 1 -- 172.21.15.17:0/4034612346 shutdown_connections 2023-07-14T12:01:13.394 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.412+0000 7f940c9ff700 1 -- 172.21.15.17:0/4034612346 wait complete. 2023-07-14T12:01:13.412 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T12:01:13.621 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.640+0000 7f277c1ff700 1 -- 172.21.15.17:0/1098889953 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2774100dd0 msgr2=0x7f2774101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:13.621 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.640+0000 7f277c1ff700 1 --2- 172.21.15.17:0/1098889953 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2774100dd0 0x7f2774101220 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2764009990 tx=0x7f2764009ca0 comp rx=0 tx=0).stop 2023-07-14T12:01:13.621 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.641+0000 7f277c1ff700 1 -- 172.21.15.17:0/1098889953 shutdown_connections 2023-07-14T12:01:13.621 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.641+0000 7f277c1ff700 1 --2- 172.21.15.17:0/1098889953 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27740ffbe0 0x7f27740ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:13.622 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.641+0000 7f277c1ff700 1 --2- 172.21.15.17:0/1098889953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2774101760 0x7f2774108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:13.622 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.641+0000 7f277c1ff700 1 --2- 172.21.15.17:0/1098889953 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2774100dd0 0x7f2774101220 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:13.622 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.641+0000 7f277c1ff700 1 -- 172.21.15.17:0/1098889953 >> 172.21.15.17:0/1098889953 conn(0x7f27740fb200 msgr2=0x7f27740fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:13.622 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.641+0000 7f277c1ff700 1 -- 172.21.15.17:0/1098889953 shutdown_connections 2023-07-14T12:01:13.622 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.641+0000 7f277c1ff700 1 -- 172.21.15.17:0/1098889953 wait complete. 2023-07-14T12:01:13.624 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.642+0000 7f277c1ff700 1 Processor -- start 2023-07-14T12:01:13.624 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.642+0000 7f277c1ff700 1 -- start start 2023-07-14T12:01:13.625 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.642+0000 7f277c1ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27740ffbe0 0x7f2774197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:13.625 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.642+0000 7f277c1ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2774100dd0 0x7f2774198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:13.625 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.642+0000 7f277c1ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2774101760 0x7f277419f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:13.625 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.642+0000 7f277c1ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277419f6d0 con 0x7f27740ffbe0 2023-07-14T12:01:13.625 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.642+0000 7f277c1ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277419f810 con 0x7f2774100dd0 2023-07-14T12:01:13.625 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.642+0000 7f277c1ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277419f9a0 con 0x7f2774101760 2023-07-14T12:01:13.626 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.643+0000 7f277979a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2774100dd0 0x7f2774198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:13.626 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.643+0000 7f2779f9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27740ffbe0 0x7f2774197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:13.626 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.643+0000 7f277979a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2774100dd0 0x7f2774198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.17:53930/0 (socket says 172.21.15.17:53930) 2023-07-14T12:01:13.626 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.643+0000 7f277979a700 1 -- 172.21.15.17:0/891152906 learned_addr learned my addr 172.21.15.17:0/891152906 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:13.626 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.643+0000 7f277a79c700 1 --2- 172.21.15.17:0/891152906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2774101760 0x7f277419f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:13.626 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.643+0000 7f277979a700 1 -- 172.21.15.17:0/891152906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2774101760 msgr2=0x7f277419f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:13.626 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.643+0000 7f277979a700 1 --2- 172.21.15.17:0/891152906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2774101760 0x7f277419f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:13.626 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.643+0000 7f277979a700 1 -- 172.21.15.17:0/891152906 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27740ffbe0 msgr2=0x7f2774197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:13.627 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.643+0000 7f277979a700 1 --2- 172.21.15.17:0/891152906 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27740ffbe0 0x7f2774197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:13.627 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.643+0000 7f277979a700 1 -- 172.21.15.17:0/891152906 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27640095f0 con 0x7f2774100dd0 2023-07-14T12:01:13.627 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.643+0000 7f2779f9b700 1 --2- 172.21.15.17:0/891152906 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27740ffbe0 0x7f2774197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T12:01:13.627 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.643+0000 7f277979a700 1 --2- 172.21.15.17:0/891152906 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2774100dd0 0x7f2774198090 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2764004760 tx=0x7f2764004790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:13.627 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.644+0000 7f276affd700 1 -- 172.21.15.17:0/891152906 <== mon.2 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276401d070 con 0x7f2774100dd0 2023-07-14T12:01:13.627 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.644+0000 7f276affd700 1 -- 172.21.15.17:0/891152906 <== mon.2 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2764004830 con 0x7f2774100dd0 2023-07-14T12:01:13.627 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.644+0000 7f277c1ff700 1 -- 172.21.15.17:0/891152906 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f277419fc20 con 0x7f2774100dd0 2023-07-14T12:01:13.628 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.644+0000 7f276affd700 1 -- 172.21.15.17:0/891152906 <== mon.2 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2764005350 con 0x7f2774100dd0 2023-07-14T12:01:13.628 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.644+0000 7f277c1ff700 1 -- 172.21.15.17:0/891152906 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27741a0040 con 0x7f2774100dd0 2023-07-14T12:01:13.628 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.645+0000 7f277c1ff700 1 -- 172.21.15.17:0/891152906 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f277410b400 con 0x7f2774100dd0 2023-07-14T12:01:13.629 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.647+0000 7f276affd700 1 -- 172.21.15.17:0/891152906 <== mon.2 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f27640054d0 con 0x7f2774100dd0 2023-07-14T12:01:13.629 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.648+0000 7f276affd700 1 --2- 172.21.15.17:0/891152906 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f27600a11d0 0x7f27600a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:13.629 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.648+0000 7f276affd700 1 -- 172.21.15.17:0/891152906 <== mon.2 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f27640ce240 con 0x7f2774100dd0 2023-07-14T12:01:13.629 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.648+0000 7f2779f9b700 1 --2- 172.21.15.17:0/891152906 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f27600a11d0 0x7f27600a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:13.629 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.649+0000 7f2779f9b700 1 --2- 172.21.15.17:0/891152906 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f27600a11d0 0x7f27600a3680 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2774100c30 tx=0x7f2770009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:13.631 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.651+0000 7f276affd700 1 -- 172.21.15.17:0/891152906 <== mon.2 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2764097670 con 0x7f2774100dd0 2023-07-14T12:01:13.839 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.859+0000 7f277c1ff700 1 -- 172.21.15.17:0/891152906 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f27741a0650 con 0x7f2774100dd0 2023-07-14T12:01:13.840 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.860+0000 7f276affd700 1 -- 172.21.15.17:0/891152906 <== mon.2 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f276409ac90 con 0x7f2774100dd0 2023-07-14T12:01:13.841 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2023-07-14T12:01:13.843 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.862+0000 7f277c1ff700 1 -- 172.21.15.17:0/891152906 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f27600a11d0 msgr2=0x7f27600a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:13.843 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.862+0000 7f277c1ff700 1 --2- 172.21.15.17:0/891152906 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f27600a11d0 0x7f27600a3680 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2774100c30 tx=0x7f2770009260 comp rx=0 tx=0).stop 2023-07-14T12:01:13.843 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.862+0000 7f277c1ff700 1 -- 172.21.15.17:0/891152906 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2774100dd0 msgr2=0x7f2774198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:13.843 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.862+0000 7f277c1ff700 1 --2- 172.21.15.17:0/891152906 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2774100dd0 0x7f2774198090 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2764004760 tx=0x7f2764004790 comp rx=0 tx=0).stop 2023-07-14T12:01:13.843 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.863+0000 7f277c1ff700 1 -- 172.21.15.17:0/891152906 shutdown_connections 2023-07-14T12:01:13.843 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.863+0000 7f277c1ff700 1 --2- 172.21.15.17:0/891152906 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f27600a11d0 0x7f27600a3680 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:13.844 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.863+0000 7f277c1ff700 1 --2- 172.21.15.17:0/891152906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2774101760 0x7f277419f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:13.844 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.863+0000 7f277c1ff700 1 --2- 172.21.15.17:0/891152906 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27740ffbe0 0x7f2774197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:13.844 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.863+0000 7f277c1ff700 1 --2- 172.21.15.17:0/891152906 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2774100dd0 0x7f2774198090 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:13.844 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.863+0000 7f277c1ff700 1 -- 172.21.15.17:0/891152906 >> 172.21.15.17:0/891152906 conn(0x7f27740fb200 msgr2=0x7f27740fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:13.844 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.863+0000 7f277c1ff700 1 -- 172.21.15.17:0/891152906 shutdown_connections 2023-07-14T12:01:13.844 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:13.863+0000 7f277c1ff700 1 -- 172.21.15.17:0/891152906 wait complete. 2023-07-14T12:01:13.862 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-14T12:01:14.069 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.088+0000 7fc0d93ff700 1 -- 172.21.15.17:0/3828096687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0d4100dd0 msgr2=0x7fc0d4101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:14.069 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.088+0000 7fc0d93ff700 1 --2- 172.21.15.17:0/3828096687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0d4100dd0 0x7fc0d4101220 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc0c8009990 tx=0x7fc0c8009ca0 comp rx=0 tx=0).stop 2023-07-14T12:01:14.069 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.088+0000 7fc0d93ff700 1 -- 172.21.15.17:0/3828096687 shutdown_connections 2023-07-14T12:01:14.069 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.088+0000 7fc0d93ff700 1 --2- 172.21.15.17:0/3828096687 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc0d40ffbe0 0x7fc0d40ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:14.069 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.088+0000 7fc0d93ff700 1 --2- 172.21.15.17:0/3828096687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0d4101760 0x7fc0d4108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:14.070 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.088+0000 7fc0d93ff700 1 --2- 172.21.15.17:0/3828096687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0d4100dd0 0x7fc0d4101220 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:14.070 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.088+0000 7fc0d93ff700 1 -- 172.21.15.17:0/3828096687 >> 172.21.15.17:0/3828096687 conn(0x7fc0d40fb200 msgr2=0x7fc0d40fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:14.070 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.089+0000 7fc0d93ff700 1 -- 172.21.15.17:0/3828096687 shutdown_connections 2023-07-14T12:01:14.070 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.089+0000 7fc0d93ff700 1 -- 172.21.15.17:0/3828096687 wait complete. 2023-07-14T12:01:14.072 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.090+0000 7fc0d93ff700 1 Processor -- start 2023-07-14T12:01:14.072 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.090+0000 7fc0d93ff700 1 -- start start 2023-07-14T12:01:14.072 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.090+0000 7fc0d93ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0d40ffbe0 0x7fc0d40742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:14.072 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.090+0000 7fc0d93ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0d4100dd0 0x7fc0d4072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:14.072 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.090+0000 7fc0d93ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc0d4101760 0x7fc0d4072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:14.072 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.090+0000 7fc0d93ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d40735e0 con 0x7fc0d4100dd0 2023-07-14T12:01:14.072 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.090+0000 7fc0d93ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d4073750 con 0x7fc0d4101760 2023-07-14T12:01:14.073 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.090+0000 7fc0d93ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d419f940 con 0x7fc0d40ffbe0 2023-07-14T12:01:14.073 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.090+0000 7fc0d27fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0d4100dd0 0x7fc0d4072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:14.073 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.090+0000 7fc0d27fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0d4100dd0 0x7fc0d4072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53426/0 (socket says 172.21.15.17:53426) 2023-07-14T12:01:14.073 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.090+0000 7fc0d27fc700 1 -- 172.21.15.17:0/3284644289 learned_addr learned my addr 172.21.15.17:0/3284644289 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:14.073 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.090+0000 7fc0d37fe700 1 --2- 172.21.15.17:0/3284644289 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc0d4101760 0x7fc0d4072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:14.073 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.090+0000 7fc0d2ffd700 1 --2- 172.21.15.17:0/3284644289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0d40ffbe0 0x7fc0d40742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:14.073 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.091+0000 7fc0d27fc700 1 -- 172.21.15.17:0/3284644289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0d40ffbe0 msgr2=0x7fc0d40742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:14.074 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.091+0000 7fc0d27fc700 1 --2- 172.21.15.17:0/3284644289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0d40ffbe0 0x7fc0d40742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:14.074 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.091+0000 7fc0d27fc700 1 -- 172.21.15.17:0/3284644289 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc0d4101760 msgr2=0x7fc0d4072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:14.074 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.091+0000 7fc0d27fc700 1 --2- 172.21.15.17:0/3284644289 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc0d4101760 0x7fc0d4072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:14.074 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.091+0000 7fc0d27fc700 1 -- 172.21.15.17:0/3284644289 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c80095f0 con 0x7fc0d4100dd0 2023-07-14T12:01:14.074 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.091+0000 7fc0d27fc700 1 --2- 172.21.15.17:0/3284644289 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0d4100dd0 0x7fc0d4072900 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc0c800b620 tx=0x7fc0c8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:14.074 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.091+0000 7fc0bbfff700 1 -- 172.21.15.17:0/3284644289 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c801d070 con 0x7fc0d4100dd0 2023-07-14T12:01:14.074 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.091+0000 7fc0d93ff700 1 -- 172.21.15.17:0/3284644289 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0d419fbc0 con 0x7fc0d4100dd0 2023-07-14T12:01:14.074 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.091+0000 7fc0bbfff700 1 -- 172.21.15.17:0/3284644289 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0c8005030 con 0x7fc0d4100dd0 2023-07-14T12:01:14.075 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.091+0000 7fc0bbfff700 1 -- 172.21.15.17:0/3284644289 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c8004b80 con 0x7fc0d4100dd0 2023-07-14T12:01:14.075 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.091+0000 7fc0d93ff700 1 -- 172.21.15.17:0/3284644289 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0d419fff0 con 0x7fc0d4100dd0 2023-07-14T12:01:14.075 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.092+0000 7fc0d93ff700 1 -- 172.21.15.17:0/3284644289 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0d4106b10 con 0x7fc0d4100dd0 2023-07-14T12:01:14.075 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.094+0000 7fc0bbfff700 1 -- 172.21.15.17:0/3284644289 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc0c80051a0 con 0x7fc0d4100dd0 2023-07-14T12:01:14.076 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.096+0000 7fc0bbfff700 1 --2- 172.21.15.17:0/3284644289 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fc0bc0a1150 0x7fc0bc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:14.076 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.096+0000 7fc0bbfff700 1 -- 172.21.15.17:0/3284644289 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fc0c80cdc60 con 0x7fc0d4100dd0 2023-07-14T12:01:14.076 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.096+0000 7fc0d2ffd700 1 --2- 172.21.15.17:0/3284644289 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fc0bc0a1150 0x7fc0bc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:14.076 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.096+0000 7fc0d2ffd700 1 --2- 172.21.15.17:0/3284644289 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fc0bc0a1150 0x7fc0bc0a3600 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4100c30 tx=0x7fc0c4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:14.079 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.100+0000 7fc0bbfff700 1 -- 172.21.15.17:0/3284644289 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc0c80970f0 con 0x7fc0d4100dd0 2023-07-14T12:01:14.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:13 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/2891646589' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T12:01:14.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:13 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/4034612346' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T12:01:14.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:13 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/2891646589' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T12:01:14.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:13 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/4034612346' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T12:01:14.143 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:13 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/2891646589' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T12:01:14.143 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:13 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/4034612346' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T12:01:14.289 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.309+0000 7fc0d93ff700 1 -- 172.21.15.17:0/3284644289 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc0d402c850 con 0x7fc0d4100dd0 2023-07-14T12:01:14.289 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.309+0000 7fc0bbfff700 1 -- 172.21.15.17:0/3284644289 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc0d402c850 con 0x7fc0d4100dd0 2023-07-14T12:01:14.289 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2023-07-14T12:01:14.292 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.311+0000 7fc0d93ff700 1 -- 172.21.15.17:0/3284644289 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fc0bc0a1150 msgr2=0x7fc0bc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:14.292 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.311+0000 7fc0d93ff700 1 --2- 172.21.15.17:0/3284644289 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fc0bc0a1150 0x7fc0bc0a3600 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4100c30 tx=0x7fc0c4009260 comp rx=0 tx=0).stop 2023-07-14T12:01:14.292 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.311+0000 7fc0d93ff700 1 -- 172.21.15.17:0/3284644289 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0d4100dd0 msgr2=0x7fc0d4072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:14.292 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.311+0000 7fc0d93ff700 1 --2- 172.21.15.17:0/3284644289 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0d4100dd0 0x7fc0d4072900 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc0c800b620 tx=0x7fc0c8004730 comp rx=0 tx=0).stop 2023-07-14T12:01:14.292 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.312+0000 7fc0d93ff700 1 -- 172.21.15.17:0/3284644289 shutdown_connections 2023-07-14T12:01:14.292 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.312+0000 7fc0d93ff700 1 --2- 172.21.15.17:0/3284644289 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fc0bc0a1150 0x7fc0bc0a3600 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:14.293 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.312+0000 7fc0d93ff700 1 --2- 172.21.15.17:0/3284644289 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc0d4101760 0x7fc0d4072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:14.293 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.312+0000 7fc0d93ff700 1 --2- 172.21.15.17:0/3284644289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0d40ffbe0 0x7fc0d40742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:14.293 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.312+0000 7fc0d93ff700 1 --2- 172.21.15.17:0/3284644289 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0d4100dd0 0x7fc0d4072900 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:14.293 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.312+0000 7fc0d93ff700 1 -- 172.21.15.17:0/3284644289 >> 172.21.15.17:0/3284644289 conn(0x7fc0d40fb200 msgr2=0x7fc0d40fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:14.293 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.312+0000 7fc0d93ff700 1 -- 172.21.15.17:0/3284644289 shutdown_connections 2023-07-14T12:01:14.293 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.312+0000 7fc0d93ff700 1 -- 172.21.15.17:0/3284644289 wait complete. 2023-07-14T12:01:14.310 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T12:01:14.515 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.535+0000 7ff5e03ff700 1 -- 172.21.15.17:0/2286621696 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff5d8100e30 msgr2=0x7ff5d81012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:14.515 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.535+0000 7ff5e03ff700 1 --2- 172.21.15.17:0/2286621696 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff5d8100e30 0x7ff5d81012a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80098f0 tx=0x7ff5c8009c00 comp rx=0 tx=0).stop 2023-07-14T12:01:14.515 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.535+0000 7ff5e03ff700 1 -- 172.21.15.17:0/2286621696 shutdown_connections 2023-07-14T12:01:14.515 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.535+0000 7ff5e03ff700 1 --2- 172.21.15.17:0/2286621696 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5d81017e0 0x7ff5d8108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:14.516 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.535+0000 7ff5e03ff700 1 --2- 172.21.15.17:0/2286621696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5d80ffba0 0x7ff5d80fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:14.516 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.535+0000 7ff5e03ff700 1 --2- 172.21.15.17:0/2286621696 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff5d8100e30 0x7ff5d81012a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:14.516 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.535+0000 7ff5e03ff700 1 -- 172.21.15.17:0/2286621696 >> 172.21.15.17:0/2286621696 conn(0x7ff5d80fb1c0 msgr2=0x7ff5d80fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:14.516 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.535+0000 7ff5e03ff700 1 -- 172.21.15.17:0/2286621696 shutdown_connections 2023-07-14T12:01:14.516 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.535+0000 7ff5e03ff700 1 -- 172.21.15.17:0/2286621696 wait complete. 2023-07-14T12:01:14.518 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.536+0000 7ff5e03ff700 1 Processor -- start 2023-07-14T12:01:14.518 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.536+0000 7ff5e03ff700 1 -- start start 2023-07-14T12:01:14.518 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.536+0000 7ff5e03ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5d80ffba0 0x7ff5d80742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:14.518 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.536+0000 7ff5e03ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff5d8100e30 0x7ff5d8072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:14.518 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.536+0000 7ff5e03ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5d81017e0 0x7ff5d8072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:14.518 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.536+0000 7ff5e03ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d80735e0 con 0x7ff5d81017e0 2023-07-14T12:01:14.519 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.536+0000 7ff5e03ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d8073750 con 0x7ff5d8100e30 2023-07-14T12:01:14.519 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.536+0000 7ff5e03ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d819f9c0 con 0x7ff5d80ffba0 2023-07-14T12:01:14.519 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.537+0000 7ff5de19b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5d80ffba0 0x7ff5d80742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:14.519 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.537+0000 7ff5de99c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5d81017e0 0x7ff5d8072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:14.519 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.537+0000 7ff5dd99a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff5d8100e30 0x7ff5d8072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:14.519 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.537+0000 7ff5de19b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5d80ffba0 0x7ff5d80742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.17:54026/0 (socket says 172.21.15.17:54026) 2023-07-14T12:01:14.519 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.537+0000 7ff5de19b700 1 -- 172.21.15.17:0/881258842 learned_addr learned my addr 172.21.15.17:0/881258842 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:14.519 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.537+0000 7ff5dd99a700 1 -- 172.21.15.17:0/881258842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5d80ffba0 msgr2=0x7ff5d80742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:14.520 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.537+0000 7ff5dd99a700 1 --2- 172.21.15.17:0/881258842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5d80ffba0 0x7ff5d80742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:14.520 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.537+0000 7ff5dd99a700 1 -- 172.21.15.17:0/881258842 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5d81017e0 msgr2=0x7ff5d8072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:14.520 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.537+0000 7ff5dd99a700 1 --2- 172.21.15.17:0/881258842 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5d81017e0 0x7ff5d8072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:14.520 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.537+0000 7ff5dd99a700 1 -- 172.21.15.17:0/881258842 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5d0009d00 con 0x7ff5d8100e30 2023-07-14T12:01:14.520 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.537+0000 7ff5de99c700 1 --2- 172.21.15.17:0/881258842 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5d81017e0 0x7ff5d8072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T12:01:14.520 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.537+0000 7ff5de19b700 1 --2- 172.21.15.17:0/881258842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5d80ffba0 0x7ff5d80742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T12:01:14.520 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.537+0000 7ff5dd99a700 1 --2- 172.21.15.17:0/881258842 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff5d8100e30 0x7ff5d8072900 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8009230 tx=0x7ff5c800b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:14.521 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.538+0000 7ff5cf7fe700 1 -- 172.21.15.17:0/881258842 <== mon.2 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c801c070 con 0x7ff5d8100e30 2023-07-14T12:01:14.521 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.538+0000 7ff5cf7fe700 1 -- 172.21.15.17:0/881258842 <== mon.2 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5c800f7d0 con 0x7ff5d8100e30 2023-07-14T12:01:14.521 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.538+0000 7ff5cf7fe700 1 -- 172.21.15.17:0/881258842 <== mon.2 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c8003a00 con 0x7ff5d8100e30 2023-07-14T12:01:14.521 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.538+0000 7ff5e03ff700 1 -- 172.21.15.17:0/881258842 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5c8009520 con 0x7ff5d8100e30 2023-07-14T12:01:14.521 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.538+0000 7ff5e03ff700 1 -- 172.21.15.17:0/881258842 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5d819fe80 con 0x7ff5d8100e30 2023-07-14T12:01:14.522 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.539+0000 7ff5e03ff700 1 -- 172.21.15.17:0/881258842 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5d810b3c0 con 0x7ff5d8100e30 2023-07-14T12:01:14.522 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.541+0000 7ff5cf7fe700 1 -- 172.21.15.17:0/881258842 <== mon.2 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff5c8020030 con 0x7ff5d8100e30 2023-07-14T12:01:14.522 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.542+0000 7ff5cf7fe700 1 --2- 172.21.15.17:0/881258842 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7ff5c40a1150 0x7ff5c40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:14.522 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.542+0000 7ff5cf7fe700 1 -- 172.21.15.17:0/881258842 <== mon.2 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7ff5c80ce1b0 con 0x7ff5d8100e30 2023-07-14T12:01:14.523 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.542+0000 7ff5de19b700 1 --2- 172.21.15.17:0/881258842 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7ff5c40a1150 0x7ff5c40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:14.523 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.543+0000 7ff5de19b700 1 --2- 172.21.15.17:0/881258842 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7ff5c40a1150 0x7ff5c40a3600 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4011550 tx=0x7ff5d4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:14.524 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.544+0000 7ff5cf7fe700 1 -- 172.21.15.17:0/881258842 <== mon.2 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff5c80975e0 con 0x7ff5d8100e30 2023-07-14T12:01:14.779 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.799+0000 7ff5e03ff700 1 -- 172.21.15.17:0/881258842 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff5d8106bb0 con 0x7ff5d8100e30 2023-07-14T12:01:14.780 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.800+0000 7ff5cf7fe700 1 -- 172.21.15.17:0/881258842 <== mon.2 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10869 (secure 0 0 0) 0x7ff5c809ac00 con 0x7ff5d8100e30 2023-07-14T12:01:14.781 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T12:01:14.781 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/55384193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":55384193},{"type":"v1","addr":"172.21.15.17:6835","nonce":55384193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/424956554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":424956554},{"type":"v1","addr":"172.21.15.17:6837","nonce":424956554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T11:59:43.292866+0000","modified":"2023-07-14T12:00:17.489445+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24565,"mds_1":24557,"mds_2":24541,"mds_3":24533,"mds_4":24517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24452":{"gid":24452,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.93:6833/326319712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":326319712},{"type":"v1","addr":"172.21.15.93:6833","nonce":326319712}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24454":{"gid":24454,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6833/878913679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":878913679},{"type":"v1","addr":"172.21.15.97:6833","nonce":878913679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.93:6835/2779420503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":2779420503},{"type":"v1","addr":"172.21.15.93:6835","nonce":2779420503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/2740799393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2740799393},{"type":"v1","addr":"172.21.15.97:6835","nonce":2740799393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.93:6837/1880673911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":1880673911},{"type":"v1","addr":"172.21.15.93:6837","nonce":1880673911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.97:6837/3684187131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3684187131},{"type":"v1","addr":"172.21.15.97:6837","nonce":3684187131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"g","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.17:6839/2976493015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":2976493015},{"type":"v1","addr":"172.21.15.17:6839","nonce":2976493015}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.97:6839/3267815331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3267815331},{"type":"v1","addr":"172.21.15.97:6839","nonce":3267815331}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.93:6839/1281342428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":1281342428},{"type":"v1","addr":"172.21.15.93:6839","nonce":1281342428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6841/3905201143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6840","nonce":3905201143},{"type":"v1","addr":"172.21.15.17:6841","nonce":3905201143}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T12:01:14.783 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.803+0000 7ff5e03ff700 1 -- 172.21.15.17:0/881258842 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7ff5c40a1150 msgr2=0x7ff5c40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:14.783 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.803+0000 7ff5e03ff700 1 --2- 172.21.15.17:0/881258842 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7ff5c40a1150 0x7ff5c40a3600 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4011550 tx=0x7ff5d4009260 comp rx=0 tx=0).stop 2023-07-14T12:01:14.783 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.803+0000 7ff5e03ff700 1 -- 172.21.15.17:0/881258842 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff5d8100e30 msgr2=0x7ff5d8072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:14.784 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.803+0000 7ff5e03ff700 1 --2- 172.21.15.17:0/881258842 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff5d8100e30 0x7ff5d8072900 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8009230 tx=0x7ff5c800b790 comp rx=0 tx=0).stop 2023-07-14T12:01:14.784 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.803+0000 7ff5e03ff700 1 -- 172.21.15.17:0/881258842 shutdown_connections 2023-07-14T12:01:14.784 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.803+0000 7ff5e03ff700 1 --2- 172.21.15.17:0/881258842 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7ff5c40a1150 0x7ff5c40a3600 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:14.784 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.803+0000 7ff5e03ff700 1 --2- 172.21.15.17:0/881258842 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5d81017e0 0x7ff5d8072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:14.784 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.803+0000 7ff5e03ff700 1 --2- 172.21.15.17:0/881258842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5d80ffba0 0x7ff5d80742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:14.784 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.803+0000 7ff5e03ff700 1 --2- 172.21.15.17:0/881258842 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff5d8100e30 0x7ff5d8072900 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:14.784 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.803+0000 7ff5e03ff700 1 -- 172.21.15.17:0/881258842 >> 172.21.15.17:0/881258842 conn(0x7ff5d80fb1c0 msgr2=0x7ff5d80fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:14.785 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.803+0000 7ff5e03ff700 1 -- 172.21.15.17:0/881258842 shutdown_connections 2023-07-14T12:01:14.785 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:14.803+0000 7ff5e03ff700 1 -- 172.21.15.17:0/881258842 wait complete. 2023-07-14T12:01:14.785 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 27 2023-07-14T12:01:14.803 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T12:01:15.009 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.028+0000 7f86847ff700 1 -- 172.21.15.17:0/266731371 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f867c100e30 msgr2=0x7f867c1012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:15.009 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.028+0000 7f86847ff700 1 --2- 172.21.15.17:0/266731371 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f867c100e30 0x7f867c1012a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f866c009990 tx=0x7f866c009ca0 comp rx=0 tx=0).stop 2023-07-14T12:01:15.010 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.028+0000 7f86847ff700 1 -- 172.21.15.17:0/266731371 shutdown_connections 2023-07-14T12:01:15.010 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.028+0000 7f86847ff700 1 --2- 172.21.15.17:0/266731371 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f867c1017e0 0x7f867c108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:15.010 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.028+0000 7f86847ff700 1 --2- 172.21.15.17:0/266731371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f867c0ffba0 0x7f867c0fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:15.010 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.028+0000 7f86847ff700 1 --2- 172.21.15.17:0/266731371 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f867c100e30 0x7f867c1012a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:15.010 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.028+0000 7f86847ff700 1 -- 172.21.15.17:0/266731371 >> 172.21.15.17:0/266731371 conn(0x7f867c0fb1c0 msgr2=0x7f867c0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:15.010 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.028+0000 7f86847ff700 1 -- 172.21.15.17:0/266731371 shutdown_connections 2023-07-14T12:01:15.011 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.028+0000 7f86847ff700 1 -- 172.21.15.17:0/266731371 wait complete. 2023-07-14T12:01:15.012 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.029+0000 7f86847ff700 1 Processor -- start 2023-07-14T12:01:15.013 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.030+0000 7f86847ff700 1 -- start start 2023-07-14T12:01:15.013 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.030+0000 7f86847ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f867c0ffba0 0x7f867c1958f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:15.013 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.030+0000 7f86847ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f867c100e30 0x7f867c195e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:15.013 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.030+0000 7f86847ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f867c1017e0 0x7f867c19cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:15.013 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.030+0000 7f86847ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867c19d470 con 0x7f867c0ffba0 2023-07-14T12:01:15.013 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.030+0000 7f86847ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867c19d5b0 con 0x7f867c100e30 2023-07-14T12:01:15.014 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.030+0000 7f86847ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867c19d740 con 0x7f867c1017e0 2023-07-14T12:01:15.014 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.030+0000 7f868259b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f867c0ffba0 0x7f867c1958f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:15.014 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.030+0000 7f868259b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f867c0ffba0 0x7f867c1958f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53452/0 (socket says 172.21.15.17:53452) 2023-07-14T12:01:15.014 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.030+0000 7f868259b700 1 -- 172.21.15.17:0/1505619594 learned_addr learned my addr 172.21.15.17:0/1505619594 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:15.014 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.030+0000 7f8681d9a700 1 --2- 172.21.15.17:0/1505619594 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f867c100e30 0x7f867c195e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:15.014 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.030+0000 7f8682d9c700 1 --2- 172.21.15.17:0/1505619594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f867c1017e0 0x7f867c19cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:15.014 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.031+0000 7f868259b700 1 -- 172.21.15.17:0/1505619594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f867c1017e0 msgr2=0x7f867c19cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:15.014 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.031+0000 7f868259b700 1 --2- 172.21.15.17:0/1505619594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f867c1017e0 0x7f867c19cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:15.015 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.031+0000 7f868259b700 1 -- 172.21.15.17:0/1505619594 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f867c100e30 msgr2=0x7f867c195e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:15.015 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.031+0000 7f868259b700 1 --2- 172.21.15.17:0/1505619594 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f867c100e30 0x7f867c195e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:15.015 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.031+0000 7f868259b700 1 -- 172.21.15.17:0/1505619594 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f866c0095f0 con 0x7f867c0ffba0 2023-07-14T12:01:15.015 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.031+0000 7f868259b700 1 --2- 172.21.15.17:0/1505619594 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f867c0ffba0 0x7f867c1958f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f867800e9a0 tx=0x7f867800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:15.015 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.031+0000 7f86737fe700 1 -- 172.21.15.17:0/1505619594 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8678009790 con 0x7f867c0ffba0 2023-07-14T12:01:15.015 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.031+0000 7f86847ff700 1 -- 172.21.15.17:0/1505619594 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f867c19da20 con 0x7f867c0ffba0 2023-07-14T12:01:15.015 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.031+0000 7f86737fe700 1 -- 172.21.15.17:0/1505619594 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8678010430 con 0x7f867c0ffba0 2023-07-14T12:01:15.016 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.031+0000 7f86737fe700 1 -- 172.21.15.17:0/1505619594 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8678018680 con 0x7f867c0ffba0 2023-07-14T12:01:15.016 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.031+0000 7f86847ff700 1 -- 172.21.15.17:0/1505619594 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f867c19df30 con 0x7f867c0ffba0 2023-07-14T12:01:15.016 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.032+0000 7f86847ff700 1 -- 172.21.15.17:0/1505619594 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f867c10b3c0 con 0x7f867c0ffba0 2023-07-14T12:01:15.017 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.034+0000 7f86737fe700 1 -- 172.21.15.17:0/1505619594 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f86780105a0 con 0x7f867c0ffba0 2023-07-14T12:01:15.017 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.035+0000 7f86737fe700 1 --2- 172.21.15.17:0/1505619594 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f86680a1150 0x7f86680a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:15.017 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.036+0000 7f86737fe700 1 -- 172.21.15.17:0/1505619594 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f8678014070 con 0x7f867c0ffba0 2023-07-14T12:01:15.017 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.036+0000 7f8681d9a700 1 --2- 172.21.15.17:0/1505619594 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f86680a1150 0x7f86680a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:15.018 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.036+0000 7f8681d9a700 1 --2- 172.21.15.17:0/1505619594 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f86680a1150 0x7f86680a3600 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f866c00b2e0 tx=0x7f866c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:15.019 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.039+0000 7f86737fe700 1 -- 172.21.15.17:0/1505619594 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8678051790 con 0x7f867c0ffba0 2023-07-14T12:01:15.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:14 smithi093 ceph-mon[128922]: pgmap v326: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s; 13 B/s, 0 objects/s recovering 2023-07-14T12:01:15.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:14 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/891152906' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T12:01:15.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:14 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/3284644289' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T12:01:15.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:14 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/881258842' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T12:01:15.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:14 smithi097 ceph-mon[129313]: pgmap v326: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s; 13 B/s, 0 objects/s recovering 2023-07-14T12:01:15.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:14 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/891152906' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T12:01:15.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:14 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/3284644289' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T12:01:15.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:14 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/881258842' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T12:01:15.143 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:14 smithi017 ceph-mon[126339]: pgmap v326: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s; 13 B/s, 0 objects/s recovering 2023-07-14T12:01:15.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:14 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/891152906' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T12:01:15.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:14 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/3284644289' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T12:01:15.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:14 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/881258842' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T12:01:15.235 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.255+0000 7f86847ff700 1 -- 172.21.15.17:0/1505619594 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f867c19e4e0 con 0x7f867c0ffba0 2023-07-14T12:01:15.237 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.257+0000 7f86737fe700 1 -- 172.21.15.17:0/1505619594 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24486 (secure 0 0 0) 0x7f86780d3080 con 0x7f867c0ffba0 2023-07-14T12:01:15.237 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T12:01:15.238 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":92,"fsid":"b3c386cc-223c-11ee-9b33-001a4aab830c","created":"2023-07-14T11:51:59.992960+0000","modified":"2023-07-14T12:01:07.806062+0000","last_up_change":"2023-07-14T11:58:55.037769+0000","last_in_change":"2023-07-14T11:58:37.114870+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:55:28.237002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:59:41.560502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T11:59:42.725461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:59:46.267029+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"57003438-a83c-4c8e-a029-56414740895c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6803","nonce":3588366384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6805","nonce":3588366384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6809","nonce":3588366384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6807","nonce":3588366384}]},"public_addr":"172.21.15.17:6803/3588366384","cluster_addr":"172.21.15.17:6805/3588366384","heartbeat_back_addr":"172.21.15.17:6809/3588366384","heartbeat_front_addr":"172.21.15.17:6807/3588366384","state":["exists","up"]},{"osd":1,"uuid":"f56b6337-d8a9-4411-b9d8-b6863dc98556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6801","nonce":3147203064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6803","nonce":3147203064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6807","nonce":3147203064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6805","nonce":3147203064}]},"public_addr":"172.21.15.93:6801/3147203064","cluster_addr":"172.21.15.93:6803/3147203064","heartbeat_back_addr":"172.21.15.93:6807/3147203064","heartbeat_front_addr":"172.21.15.93:6805/3147203064","state":["exists","up"]},{"osd":2,"uuid":"fc34bb7f-551f-46e1-99c0-59924d6c1346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6801","nonce":1877269815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6803","nonce":1877269815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6807","nonce":1877269815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6805","nonce":1877269815}]},"public_addr":"172.21.15.97:6801/1877269815","cluster_addr":"172.21.15.97:6803/1877269815","heartbeat_back_addr":"172.21.15.97:6807/1877269815","heartbeat_front_addr":"172.21.15.97:6805/1877269815","state":["exists","up"]},{"osd":3,"uuid":"37a1583c-4fb8-412a-a0c8-f83b1ff26a91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6811","nonce":2784619821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6813","nonce":2784619821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6817","nonce":2784619821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6815","nonce":2784619821}]},"public_addr":"172.21.15.17:6811/2784619821","cluster_addr":"172.21.15.17:6813/2784619821","heartbeat_back_addr":"172.21.15.17:6817/2784619821","heartbeat_front_addr":"172.21.15.17:6815/2784619821","state":["exists","up"]},{"osd":4,"uuid":"f4b9f377-d519-4b14-ac4a-f1debaeed9f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6809","nonce":3151801173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6811","nonce":3151801173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6815","nonce":3151801173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6813","nonce":3151801173}]},"public_addr":"172.21.15.93:6809/3151801173","cluster_addr":"172.21.15.93:6811/3151801173","heartbeat_back_addr":"172.21.15.93:6815/3151801173","heartbeat_front_addr":"172.21.15.93:6813/3151801173","state":["exists","up"]},{"osd":5,"uuid":"2d2af354-0535-4ef9-b166-3ef219498db8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6809","nonce":2184413924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6811","nonce":2184413924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6815","nonce":2184413924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6813","nonce":2184413924}]},"public_addr":"172.21.15.97:6809/2184413924","cluster_addr":"172.21.15.97:6811/2184413924","heartbeat_back_addr":"172.21.15.97:6815/2184413924","heartbeat_front_addr":"172.21.15.97:6813/2184413924","state":["exists","up"]},{"osd":6,"uuid":"9cdf0c4b-62dc-4168-864e-a9d1650a9ea8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6819","nonce":50486581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6821","nonce":50486581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6825","nonce":50486581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6823","nonce":50486581}]},"public_addr":"172.21.15.17:6819/50486581","cluster_addr":"172.21.15.17:6821/50486581","heartbeat_back_addr":"172.21.15.17:6825/50486581","heartbeat_front_addr":"172.21.15.17:6823/50486581","state":["exists","up"]},{"osd":7,"uuid":"09f60823-5491-45cd-9324-6ae0d56ce262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6817","nonce":2596912392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6819","nonce":2596912392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6823","nonce":2596912392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6821","nonce":2596912392}]},"public_addr":"172.21.15.93:6817/2596912392","cluster_addr":"172.21.15.93:6819/2596912392","heartbeat_back_addr":"172.21.15.93:6823/2596912392","heartbeat_front_addr":"172.21.15.93:6821/2596912392","state":["exists","up"]},{"osd":8,"uuid":"8c5771c9-718e-4313-a2b6-5e1c37c53f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6817","nonce":1965492859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6819","nonce":1965492859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6823","nonce":1965492859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6821","nonce":1965492859}]},"public_addr":"172.21.15.97:6817/1965492859","cluster_addr":"172.21.15.97:6819/1965492859","heartbeat_back_addr":"172.21.15.97:6823/1965492859","heartbeat_front_addr":"172.21.15.97:6821/1965492859","state":["exists","up"]},{"osd":9,"uuid":"1197b8a3-c09f-4c57-aba1-9e988a1620b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6827","nonce":2512645700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6829","nonce":2512645700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6833","nonce":2512645700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6831","nonce":2512645700}]},"public_addr":"172.21.15.17:6827/2512645700","cluster_addr":"172.21.15.17:6829/2512645700","heartbeat_back_addr":"172.21.15.17:6833/2512645700","heartbeat_front_addr":"172.21.15.17:6831/2512645700","state":["exists","up"]},{"osd":10,"uuid":"f99918c2-cdd8-4a7a-be2e-d1d6e0f828a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6825","nonce":264429558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6827","nonce":264429558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6831","nonce":264429558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6829","nonce":264429558}]},"public_addr":"172.21.15.93:6825/264429558","cluster_addr":"172.21.15.93:6827/264429558","heartbeat_back_addr":"172.21.15.93:6831/264429558","heartbeat_front_addr":"172.21.15.93:6829/264429558","state":["exists","up"]},{"osd":11,"uuid":"6bbb5b79-8d4d-441f-9583-203df6c78cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6825","nonce":3389913754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6827","nonce":3389913754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6831","nonce":3389913754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6829","nonce":3389913754}]},"public_addr":"172.21.15.97:6825/3389913754","cluster_addr":"172.21.15.97:6827/3389913754","heartbeat_back_addr":"172.21.15.97:6831/3389913754","heartbeat_front_addr":"172.21.15.97:6829/3389913754","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:54:41.079618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:03.207899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:24.136961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:50.392609+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:56:12.136237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:56:33.212314+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:00.243646+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:22.059234+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:43.232399+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:10.043896+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:31.910780+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:53.158983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.9","mappings":[{"from":5,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/2084033276":"2023-07-15T11:53:05.639597+0000","172.21.15.17:6801/1059284716":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/3651785636":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/1373538024":"2023-07-15T11:52:37.887946+0000","172.21.15.17:0/3560395020":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6800/1059284716":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/4262912861":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/2258375202":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6800/2804227742":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6801/2804227742":"2023-07-15T11:52:37.887946+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T12:01:15.240 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.259+0000 7f86847ff700 1 -- 172.21.15.17:0/1505619594 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f86680a1150 msgr2=0x7f86680a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:15.240 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.260+0000 7f86847ff700 1 --2- 172.21.15.17:0/1505619594 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f86680a1150 0x7f86680a3600 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f866c00b2e0 tx=0x7f866c005fb0 comp rx=0 tx=0).stop 2023-07-14T12:01:15.240 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.260+0000 7f86847ff700 1 -- 172.21.15.17:0/1505619594 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f867c0ffba0 msgr2=0x7f867c1958f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:15.240 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.260+0000 7f86847ff700 1 --2- 172.21.15.17:0/1505619594 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f867c0ffba0 0x7f867c1958f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f867800e9a0 tx=0x7f867800ecb0 comp rx=0 tx=0).stop 2023-07-14T12:01:15.240 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.260+0000 7f86847ff700 1 -- 172.21.15.17:0/1505619594 shutdown_connections 2023-07-14T12:01:15.240 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.260+0000 7f86847ff700 1 --2- 172.21.15.17:0/1505619594 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f86680a1150 0x7f86680a3600 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:15.241 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.260+0000 7f86847ff700 1 --2- 172.21.15.17:0/1505619594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f867c1017e0 0x7f867c19cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:15.241 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.260+0000 7f86847ff700 1 --2- 172.21.15.17:0/1505619594 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f867c0ffba0 0x7f867c1958f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:15.241 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.260+0000 7f86847ff700 1 --2- 172.21.15.17:0/1505619594 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f867c100e30 0x7f867c195e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:15.241 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.260+0000 7f86847ff700 1 -- 172.21.15.17:0/1505619594 >> 172.21.15.17:0/1505619594 conn(0x7f867c0fb1c0 msgr2=0x7f867c0fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:15.241 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.260+0000 7f86847ff700 1 -- 172.21.15.17:0/1505619594 shutdown_connections 2023-07-14T12:01:15.241 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.260+0000 7f86847ff700 1 -- 172.21.15.17:0/1505619594 wait complete. 2023-07-14T12:01:15.259 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T11:55:28.237002+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T11:59:41.560502+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T11:59:42.725461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-14T11:59:46.267029+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-14T12:01:15.261 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T12:01:15.468 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.488+0000 7fc8b49ff700 1 -- 172.21.15.17:0/345018972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8b00ffbe0 msgr2=0x7fc8b00ffff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:15.468 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.488+0000 7fc8b49ff700 1 --2- 172.21.15.17:0/345018972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8b00ffbe0 0x7fc8b00ffff0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0009990 tx=0x7fc8a0009ca0 comp rx=0 tx=0).stop 2023-07-14T12:01:15.469 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.488+0000 7fc8b49ff700 1 -- 172.21.15.17:0/345018972 shutdown_connections 2023-07-14T12:01:15.469 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.488+0000 7fc8b49ff700 1 --2- 172.21.15.17:0/345018972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8b0101760 0x7fc8b0108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:15.469 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.488+0000 7fc8b49ff700 1 --2- 172.21.15.17:0/345018972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8b00ffbe0 0x7fc8b00ffff0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:15.469 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.488+0000 7fc8b49ff700 1 --2- 172.21.15.17:0/345018972 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc8b0100dd0 0x7fc8b0101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:15.469 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.488+0000 7fc8b49ff700 1 -- 172.21.15.17:0/345018972 >> 172.21.15.17:0/345018972 conn(0x7fc8b00fb200 msgr2=0x7fc8b00fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:15.469 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.488+0000 7fc8b49ff700 1 -- 172.21.15.17:0/345018972 shutdown_connections 2023-07-14T12:01:15.469 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.488+0000 7fc8b49ff700 1 -- 172.21.15.17:0/345018972 wait complete. 2023-07-14T12:01:15.470 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.489+0000 7fc8b49ff700 1 Processor -- start 2023-07-14T12:01:15.471 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.489+0000 7fc8b49ff700 1 -- start start 2023-07-14T12:01:15.471 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.490+0000 7fc8b49ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8b00ffbe0 0x7fc8b00742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:15.471 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.490+0000 7fc8b49ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8b0100dd0 0x7fc8b0072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:15.471 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.490+0000 7fc8b49ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc8b0101760 0x7fc8b0072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:15.471 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.490+0000 7fc8b49ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b00735e0 con 0x7fc8b00ffbe0 2023-07-14T12:01:15.471 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.490+0000 7fc8b49ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b0073750 con 0x7fc8b0101760 2023-07-14T12:01:15.471 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.490+0000 7fc8b49ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b019fa20 con 0x7fc8b0100dd0 2023-07-14T12:01:15.472 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.490+0000 7fc8ae59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8b00ffbe0 0x7fc8b00742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:15.472 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.490+0000 7fc8add9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8b0100dd0 0x7fc8b0072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:15.472 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.490+0000 7fc8ae59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8b00ffbe0 0x7fc8b00742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53466/0 (socket says 172.21.15.17:53466) 2023-07-14T12:01:15.472 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.490+0000 7fc8ae59c700 1 -- 172.21.15.17:0/3956070411 learned_addr learned my addr 172.21.15.17:0/3956070411 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:15.472 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.490+0000 7fc8aed9d700 1 --2- 172.21.15.17:0/3956070411 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc8b0101760 0x7fc8b0072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:15.472 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.490+0000 7fc8ae59c700 1 -- 172.21.15.17:0/3956070411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8b0100dd0 msgr2=0x7fc8b0072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:15.472 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.490+0000 7fc8ae59c700 1 --2- 172.21.15.17:0/3956070411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8b0100dd0 0x7fc8b0072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:15.473 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.490+0000 7fc8ae59c700 1 -- 172.21.15.17:0/3956070411 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc8b0101760 msgr2=0x7fc8b0072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:15.473 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.490+0000 7fc8ae59c700 1 --2- 172.21.15.17:0/3956070411 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc8b0101760 0x7fc8b0072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:15.473 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.490+0000 7fc8ae59c700 1 -- 172.21.15.17:0/3956070411 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a00095f0 con 0x7fc8b00ffbe0 2023-07-14T12:01:15.473 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.490+0000 7fc8add9b700 1 --2- 172.21.15.17:0/3956070411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8b0100dd0 0x7fc8b0072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T12:01:15.474 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.491+0000 7fc8ae59c700 1 --2- 172.21.15.17:0/3956070411 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8b00ffbe0 0x7fc8b00742c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc8a000b620 tx=0x7fc8a0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:15.474 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.491+0000 7fc89f7fe700 1 -- 172.21.15.17:0/3956070411 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a001d070 con 0x7fc8b00ffbe0 2023-07-14T12:01:15.474 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.491+0000 7fc8b49ff700 1 -- 172.21.15.17:0/3956070411 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8b019fca0 con 0x7fc8b00ffbe0 2023-07-14T12:01:15.474 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.491+0000 7fc89f7fe700 1 -- 172.21.15.17:0/3956070411 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8a0021410 con 0x7fc8b00ffbe0 2023-07-14T12:01:15.474 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.491+0000 7fc89f7fe700 1 -- 172.21.15.17:0/3956070411 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a0017650 con 0x7fc8b00ffbe0 2023-07-14T12:01:15.475 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.491+0000 7fc8b49ff700 1 -- 172.21.15.17:0/3956070411 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b01a00c0 con 0x7fc8b00ffbe0 2023-07-14T12:01:15.475 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.492+0000 7fc8b49ff700 1 -- 172.21.15.17:0/3956070411 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8b0107170 con 0x7fc8b00ffbe0 2023-07-14T12:01:15.475 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.494+0000 7fc89f7fe700 1 -- 172.21.15.17:0/3956070411 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc8a0021580 con 0x7fc8b00ffbe0 2023-07-14T12:01:15.476 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.496+0000 7fc89f7fe700 1 --2- 172.21.15.17:0/3956070411 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fc8980a1150 0x7fc8980a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:15.476 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.496+0000 7fc89f7fe700 1 -- 172.21.15.17:0/3956070411 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fc8a00ce7a0 con 0x7fc8b00ffbe0 2023-07-14T12:01:15.476 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.496+0000 7fc8add9b700 1 --2- 172.21.15.17:0/3956070411 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fc8980a1150 0x7fc8980a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:15.477 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.497+0000 7fc8add9b700 1 --2- 172.21.15.17:0/3956070411 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fc8980a1150 0x7fc8980a3600 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4011490 tx=0x7fc8a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:15.478 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.498+0000 7fc89f7fe700 1 -- 172.21.15.17:0/3956070411 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc8a0097c30 con 0x7fc8b00ffbe0 2023-07-14T12:01:15.685 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.705+0000 7fc8b49ff700 1 -- 172.21.15.17:0/3956070411 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc8b002ccd0 con 0x7fc8b00ffbe0 2023-07-14T12:01:15.686 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.705+0000 7fc89f7fe700 1 -- 172.21.15.17:0/3956070411 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc8b002ccd0 con 0x7fc8b00ffbe0 2023-07-14T12:01:15.686 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2023-07-14T12:01:15.688 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.708+0000 7fc8b49ff700 1 -- 172.21.15.17:0/3956070411 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fc8980a1150 msgr2=0x7fc8980a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:15.688 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.708+0000 7fc8b49ff700 1 --2- 172.21.15.17:0/3956070411 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fc8980a1150 0x7fc8980a3600 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4011490 tx=0x7fc8a4009260 comp rx=0 tx=0).stop 2023-07-14T12:01:15.688 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.708+0000 7fc8b49ff700 1 -- 172.21.15.17:0/3956070411 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8b00ffbe0 msgr2=0x7fc8b00742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:15.688 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.708+0000 7fc8b49ff700 1 --2- 172.21.15.17:0/3956070411 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8b00ffbe0 0x7fc8b00742c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc8a000b620 tx=0x7fc8a0005740 comp rx=0 tx=0).stop 2023-07-14T12:01:15.688 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.708+0000 7fc8b49ff700 1 -- 172.21.15.17:0/3956070411 shutdown_connections 2023-07-14T12:01:15.689 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.708+0000 7fc8b49ff700 1 --2- 172.21.15.17:0/3956070411 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fc8980a1150 0x7fc8980a3600 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:15.689 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.708+0000 7fc8b49ff700 1 --2- 172.21.15.17:0/3956070411 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc8b0101760 0x7fc8b0072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:15.689 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.708+0000 7fc8b49ff700 1 --2- 172.21.15.17:0/3956070411 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8b00ffbe0 0x7fc8b00742c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:15.689 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.708+0000 7fc8b49ff700 1 --2- 172.21.15.17:0/3956070411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8b0100dd0 0x7fc8b0072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:15.689 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.708+0000 7fc8b49ff700 1 -- 172.21.15.17:0/3956070411 >> 172.21.15.17:0/3956070411 conn(0x7fc8b00fb200 msgr2=0x7fc8b01042b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:15.689 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.708+0000 7fc8b49ff700 1 -- 172.21.15.17:0/3956070411 shutdown_connections 2023-07-14T12:01:15.689 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.708+0000 7fc8b49ff700 1 -- 172.21.15.17:0/3956070411 wait complete. 2023-07-14T12:01:15.706 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T12:01:15.920 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.939+0000 7f89149ff700 1 -- 172.21.15.17:0/400340818 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8910101800 msgr2=0x7f89101080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:15.920 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.939+0000 7f89149ff700 1 --2- 172.21.15.17:0/400340818 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8910101800 0x7f89101080a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f890800b0a0 tx=0x7f890800b3b0 comp rx=0 tx=0).stop 2023-07-14T12:01:15.920 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.939+0000 7f89149ff700 1 -- 172.21.15.17:0/400340818 shutdown_connections 2023-07-14T12:01:15.921 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.939+0000 7f89149ff700 1 --2- 172.21.15.17:0/400340818 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f89100ffbc0 0x7f89100fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:15.921 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.939+0000 7f89149ff700 1 --2- 172.21.15.17:0/400340818 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8910101800 0x7f89101080a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:15.921 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.939+0000 7f89149ff700 1 --2- 172.21.15.17:0/400340818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8910100e50 0x7f89101012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:15.921 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.939+0000 7f89149ff700 1 -- 172.21.15.17:0/400340818 >> 172.21.15.17:0/400340818 conn(0x7f89100fb200 msgr2=0x7f89100fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:15.921 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.939+0000 7f89149ff700 1 -- 172.21.15.17:0/400340818 shutdown_connections 2023-07-14T12:01:15.921 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.939+0000 7f89149ff700 1 -- 172.21.15.17:0/400340818 wait complete. 2023-07-14T12:01:15.923 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.940+0000 7f89149ff700 1 Processor -- start 2023-07-14T12:01:15.924 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.940+0000 7f89149ff700 1 -- start start 2023-07-14T12:01:15.924 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.941+0000 7f89149ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89100ffbc0 0x7f89100742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:15.924 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.941+0000 7f89149ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8910100e50 0x7f8910072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:15.924 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.941+0000 7f89149ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8910101800 0x7f8910072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:15.924 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.941+0000 7f89149ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89100735e0 con 0x7f89100ffbc0 2023-07-14T12:01:15.924 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.941+0000 7f89149ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8910073750 con 0x7f8910100e50 2023-07-14T12:01:15.924 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.941+0000 7f89149ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891019fa50 con 0x7f8910101800 2023-07-14T12:01:15.925 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.941+0000 7f890e59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89100ffbc0 0x7f89100742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:15.925 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.941+0000 7f890e59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89100ffbc0 0x7f89100742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53498/0 (socket says 172.21.15.17:53498) 2023-07-14T12:01:15.925 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.941+0000 7f890e59c700 1 -- 172.21.15.17:0/2660906681 learned_addr learned my addr 172.21.15.17:0/2660906681 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:15.925 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.941+0000 7f890dd9b700 1 --2- 172.21.15.17:0/2660906681 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8910100e50 0x7f8910072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:15.925 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.941+0000 7f890ed9d700 1 --2- 172.21.15.17:0/2660906681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8910101800 0x7f8910072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:15.925 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.941+0000 7f890e59c700 1 -- 172.21.15.17:0/2660906681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8910101800 msgr2=0x7f8910072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:15.926 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.941+0000 7f890e59c700 1 --2- 172.21.15.17:0/2660906681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8910101800 0x7f8910072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:15.926 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.941+0000 7f890e59c700 1 -- 172.21.15.17:0/2660906681 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8910100e50 msgr2=0x7f8910072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:15.926 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.941+0000 7f890e59c700 1 --2- 172.21.15.17:0/2660906681 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8910100e50 0x7f8910072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:15.926 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.942+0000 7f890e59c700 1 -- 172.21.15.17:0/2660906681 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8908009d00 con 0x7f89100ffbc0 2023-07-14T12:01:15.926 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.942+0000 7f890e59c700 1 --2- 172.21.15.17:0/2660906681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89100ffbc0 0x7f89100742c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f890000e9a0 tx=0x7f890000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:15.926 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.942+0000 7f88ff7fe700 1 -- 172.21.15.17:0/2660906681 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8900009790 con 0x7f89100ffbc0 2023-07-14T12:01:15.926 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.942+0000 7f88ff7fe700 1 -- 172.21.15.17:0/2660906681 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f890000cbd0 con 0x7f89100ffbc0 2023-07-14T12:01:15.927 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.942+0000 7f89149ff700 1 -- 172.21.15.17:0/2660906681 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f891019fd30 con 0x7f89100ffbc0 2023-07-14T12:01:15.927 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.942+0000 7f88ff7fe700 1 -- 172.21.15.17:0/2660906681 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8900018680 con 0x7f89100ffbc0 2023-07-14T12:01:15.927 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.942+0000 7f89149ff700 1 -- 172.21.15.17:0/2660906681 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89101a0190 con 0x7f89100ffbc0 2023-07-14T12:01:15.927 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.943+0000 7f89149ff700 1 -- 172.21.15.17:0/2660906681 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8910106aa0 con 0x7f89100ffbc0 2023-07-14T12:01:15.928 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.945+0000 7f88ff7fe700 1 -- 172.21.15.17:0/2660906681 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f890000cd40 con 0x7f89100ffbc0 2023-07-14T12:01:15.928 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.947+0000 7f88ff7fe700 1 --2- 172.21.15.17:0/2660906681 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f88f80a1150 0x7f88f80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:15.928 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.947+0000 7f88ff7fe700 1 -- 172.21.15.17:0/2660906681 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f8900014070 con 0x7f89100ffbc0 2023-07-14T12:01:15.928 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.947+0000 7f890dd9b700 1 --2- 172.21.15.17:0/2660906681 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f88f80a1150 0x7f88f80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:15.929 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.948+0000 7f890dd9b700 1 --2- 172.21.15.17:0/2660906681 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f88f80a1150 0x7f88f80a3600 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8904009c60 tx=0x7f8904009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:15.930 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:15.950+0000 7f88ff7fe700 1 -- 172.21.15.17:0/2660906681 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8900051790 con 0x7f89100ffbc0 2023-07-14T12:01:16.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:15 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/1505619594' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T12:01:16.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:15 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/3956070411' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T12:01:16.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:15 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/1505619594' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T12:01:16.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:15 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/3956070411' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T12:01:16.137 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.158+0000 7f89149ff700 1 -- 172.21.15.17:0/2660906681 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f89100611c0 con 0x7f89100ffbc0 2023-07-14T12:01:16.138 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.158+0000 7f88ff7fe700 1 -- 172.21.15.17:0/2660906681 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f89000962b0 con 0x7f89100ffbc0 2023-07-14T12:01:16.138 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2023-07-14T12:01:16.140 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.160+0000 7f89149ff700 1 -- 172.21.15.17:0/2660906681 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f88f80a1150 msgr2=0x7f88f80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:16.141 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.160+0000 7f89149ff700 1 --2- 172.21.15.17:0/2660906681 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f88f80a1150 0x7f88f80a3600 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8904009c60 tx=0x7f8904009260 comp rx=0 tx=0).stop 2023-07-14T12:01:16.141 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.160+0000 7f89149ff700 1 -- 172.21.15.17:0/2660906681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89100ffbc0 msgr2=0x7f89100742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:16.141 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.160+0000 7f89149ff700 1 --2- 172.21.15.17:0/2660906681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89100ffbc0 0x7f89100742c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f890000e9a0 tx=0x7f890000ecb0 comp rx=0 tx=0).stop 2023-07-14T12:01:16.141 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.160+0000 7f89149ff700 1 -- 172.21.15.17:0/2660906681 shutdown_connections 2023-07-14T12:01:16.141 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.160+0000 7f89149ff700 1 --2- 172.21.15.17:0/2660906681 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f88f80a1150 0x7f88f80a3600 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:16.141 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.160+0000 7f89149ff700 1 --2- 172.21.15.17:0/2660906681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8910101800 0x7f8910072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:16.142 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.160+0000 7f89149ff700 1 --2- 172.21.15.17:0/2660906681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89100ffbc0 0x7f89100742c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:16.142 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.160+0000 7f89149ff700 1 --2- 172.21.15.17:0/2660906681 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8910100e50 0x7f8910072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:16.142 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.160+0000 7f89149ff700 1 -- 172.21.15.17:0/2660906681 >> 172.21.15.17:0/2660906681 conn(0x7f89100fb200 msgr2=0x7f89100fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:16.142 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.161+0000 7f89149ff700 1 -- 172.21.15.17:0/2660906681 shutdown_connections 2023-07-14T12:01:16.142 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.161+0000 7f89149ff700 1 -- 172.21.15.17:0/2660906681 wait complete. 2023-07-14T12:01:16.143 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:15 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/1505619594' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T12:01:16.143 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:15 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/3956070411' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T12:01:16.159 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T12:01:16.367 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.387+0000 7f4cf95ff700 1 -- 172.21.15.17:0/2700386194 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4cf4101760 msgr2=0x7f4cf4108020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:16.368 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.387+0000 7f4cf95ff700 1 --2- 172.21.15.17:0/2700386194 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4cf4101760 0x7f4cf4108020 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4cec00b0a0 tx=0x7f4cec00b3b0 comp rx=0 tx=0).stop 2023-07-14T12:01:16.368 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.387+0000 7f4cf95ff700 1 -- 172.21.15.17:0/2700386194 shutdown_connections 2023-07-14T12:01:16.368 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.387+0000 7f4cf95ff700 1 --2- 172.21.15.17:0/2700386194 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4cf4101760 0x7f4cf4108020 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:16.368 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.387+0000 7f4cf95ff700 1 --2- 172.21.15.17:0/2700386194 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cf40ffbe0 0x7f4cf40ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:16.369 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.387+0000 7f4cf95ff700 1 --2- 172.21.15.17:0/2700386194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cf4100dd0 0x7f4cf4101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:16.369 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.387+0000 7f4cf95ff700 1 -- 172.21.15.17:0/2700386194 >> 172.21.15.17:0/2700386194 conn(0x7f4cf40fb200 msgr2=0x7f4cf40fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:16.369 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.387+0000 7f4cf95ff700 1 -- 172.21.15.17:0/2700386194 shutdown_connections 2023-07-14T12:01:16.369 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.387+0000 7f4cf95ff700 1 -- 172.21.15.17:0/2700386194 wait complete. 2023-07-14T12:01:16.371 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.388+0000 7f4cf95ff700 1 Processor -- start 2023-07-14T12:01:16.371 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.388+0000 7f4cf95ff700 1 -- start start 2023-07-14T12:01:16.371 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.389+0000 7f4cf95ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cf40ffbe0 0x7f4cf4195990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:16.371 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.389+0000 7f4cf95ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4cf4100dd0 0x7f4cf4195ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:16.371 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.389+0000 7f4cf95ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cf4101760 0x7f4cf419cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:16.372 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.389+0000 7f4cf95ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf419d510 con 0x7f4cf40ffbe0 2023-07-14T12:01:16.372 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.389+0000 7f4cf95ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf419d650 con 0x7f4cf4100dd0 2023-07-14T12:01:16.372 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.389+0000 7f4cf95ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf419d7e0 con 0x7f4cf4101760 2023-07-14T12:01:16.372 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.389+0000 7f4cf2ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cf40ffbe0 0x7f4cf4195990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:16.372 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.389+0000 7f4cf27fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4cf4100dd0 0x7f4cf4195ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:16.372 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.389+0000 7f4cf2ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cf40ffbe0 0x7f4cf4195990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53520/0 (socket says 172.21.15.17:53520) 2023-07-14T12:01:16.373 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.389+0000 7f4cf2ffd700 1 -- 172.21.15.17:0/1363131501 learned_addr learned my addr 172.21.15.17:0/1363131501 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:16.373 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.389+0000 7f4cf37fe700 1 --2- 172.21.15.17:0/1363131501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cf4101760 0x7f4cf419cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:16.373 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.389+0000 7f4cf27fc700 1 -- 172.21.15.17:0/1363131501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cf4101760 msgr2=0x7f4cf419cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:16.373 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.389+0000 7f4cf27fc700 1 --2- 172.21.15.17:0/1363131501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cf4101760 0x7f4cf419cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:16.374 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.389+0000 7f4cf27fc700 1 -- 172.21.15.17:0/1363131501 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cf40ffbe0 msgr2=0x7f4cf4195990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:16.374 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.389+0000 7f4cf27fc700 1 --2- 172.21.15.17:0/1363131501 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cf40ffbe0 0x7f4cf4195990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:16.374 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.389+0000 7f4cf27fc700 1 -- 172.21.15.17:0/1363131501 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce4009520 con 0x7f4cf4100dd0 2023-07-14T12:01:16.374 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.389+0000 7f4cf2ffd700 1 --2- 172.21.15.17:0/1363131501 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cf40ffbe0 0x7f4cf4195990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T12:01:16.374 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.390+0000 7f4cf27fc700 1 --2- 172.21.15.17:0/1363131501 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4cf4100dd0 0x7f4cf4195ed0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4ce800e9f0 tx=0x7f4ce800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:16.374 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.390+0000 7f4cdbfff700 1 -- 172.21.15.17:0/1363131501 <== mon.2 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce800c9c0 con 0x7f4cf4100dd0 2023-07-14T12:01:16.375 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.390+0000 7f4cdbfff700 1 -- 172.21.15.17:0/1363131501 <== mon.2 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ce8010430 con 0x7f4cf4100dd0 2023-07-14T12:01:16.375 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.390+0000 7f4cf95ff700 1 -- 172.21.15.17:0/1363131501 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cec009d00 con 0x7f4cf4100dd0 2023-07-14T12:01:16.375 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.390+0000 7f4cdbfff700 1 -- 172.21.15.17:0/1363131501 <== mon.2 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8018680 con 0x7f4cf4100dd0 2023-07-14T12:01:16.375 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.390+0000 7f4cf95ff700 1 -- 172.21.15.17:0/1363131501 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cf419de40 con 0x7f4cf4100dd0 2023-07-14T12:01:16.376 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.391+0000 7f4cf95ff700 1 -- 172.21.15.17:0/1363131501 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cf41070a0 con 0x7f4cf4100dd0 2023-07-14T12:01:16.376 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.393+0000 7f4cdbfff700 1 -- 172.21.15.17:0/1363131501 <== mon.2 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4ce8010860 con 0x7f4cf4100dd0 2023-07-14T12:01:16.376 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.395+0000 7f4cdbfff700 1 --2- 172.21.15.17:0/1363131501 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f4cdc0a1150 0x7f4cdc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:16.376 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.395+0000 7f4cdbfff700 1 -- 172.21.15.17:0/1363131501 <== mon.2 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f4ce8014070 con 0x7f4cf4100dd0 2023-07-14T12:01:16.377 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.395+0000 7f4cf2ffd700 1 --2- 172.21.15.17:0/1363131501 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f4cdc0a1150 0x7f4cdc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:16.377 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.395+0000 7f4cf2ffd700 1 --2- 172.21.15.17:0/1363131501 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f4cdc0a1150 0x7f4cdc0a3600 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4009fd0 tx=0x7f4ce4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:16.377 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.397+0000 7f4cdbfff700 1 -- 172.21.15.17:0/1363131501 <== mon.2 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4ce8096340 con 0x7f4cf4100dd0 2023-07-14T12:01:16.584 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.604+0000 7f4cf95ff700 1 -- 172.21.15.17:0/1363131501 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4cf402ca60 con 0x7f4cf4100dd0 2023-07-14T12:01:16.585 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.605+0000 7f4cdbfff700 1 -- 172.21.15.17:0/1363131501 <== mon.2 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4ce8026020 con 0x7f4cf4100dd0 2023-07-14T12:01:16.585 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2023-07-14T12:01:16.588 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.607+0000 7f4cf95ff700 1 -- 172.21.15.17:0/1363131501 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f4cdc0a1150 msgr2=0x7f4cdc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:16.588 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.607+0000 7f4cf95ff700 1 --2- 172.21.15.17:0/1363131501 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f4cdc0a1150 0x7f4cdc0a3600 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4009fd0 tx=0x7f4ce4009260 comp rx=0 tx=0).stop 2023-07-14T12:01:16.588 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.607+0000 7f4cf95ff700 1 -- 172.21.15.17:0/1363131501 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4cf4100dd0 msgr2=0x7f4cf4195ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:16.588 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.607+0000 7f4cf95ff700 1 --2- 172.21.15.17:0/1363131501 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4cf4100dd0 0x7f4cf4195ed0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4ce800e9f0 tx=0x7f4ce800ed00 comp rx=0 tx=0).stop 2023-07-14T12:01:16.588 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.607+0000 7f4cf95ff700 1 -- 172.21.15.17:0/1363131501 shutdown_connections 2023-07-14T12:01:16.588 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.607+0000 7f4cf95ff700 1 --2- 172.21.15.17:0/1363131501 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f4cdc0a1150 0x7f4cdc0a3600 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:16.588 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.607+0000 7f4cf95ff700 1 --2- 172.21.15.17:0/1363131501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cf4101760 0x7f4cf419cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:16.589 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.607+0000 7f4cf95ff700 1 --2- 172.21.15.17:0/1363131501 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cf40ffbe0 0x7f4cf4195990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:16.589 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.607+0000 7f4cf95ff700 1 --2- 172.21.15.17:0/1363131501 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4cf4100dd0 0x7f4cf4195ed0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:16.589 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.608+0000 7f4cf95ff700 1 -- 172.21.15.17:0/1363131501 >> 172.21.15.17:0/1363131501 conn(0x7f4cf40fb200 msgr2=0x7f4cf41041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:16.589 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.608+0000 7f4cf95ff700 1 -- 172.21.15.17:0/1363131501 shutdown_connections 2023-07-14T12:01:16.589 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.608+0000 7f4cf95ff700 1 -- 172.21.15.17:0/1363131501 wait complete. 2023-07-14T12:01:16.606 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-14T12:01:16.814 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.834+0000 7ff16b9ff700 1 -- 172.21.15.17:0/244612129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff164100e50 msgr2=0x7ff1641012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:16.814 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.834+0000 7ff16b9ff700 1 --2- 172.21.15.17:0/244612129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff164100e50 0x7ff1641012c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff154009990 tx=0x7ff154009ca0 comp rx=0 tx=0).stop 2023-07-14T12:01:16.814 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.834+0000 7ff16b9ff700 1 -- 172.21.15.17:0/244612129 shutdown_connections 2023-07-14T12:01:16.815 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.834+0000 7ff16b9ff700 1 --2- 172.21.15.17:0/244612129 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1640ffbc0 0x7ff1640fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:16.815 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.834+0000 7ff16b9ff700 1 --2- 172.21.15.17:0/244612129 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff164101800 0x7ff1641080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:16.815 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.834+0000 7ff16b9ff700 1 --2- 172.21.15.17:0/244612129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff164100e50 0x7ff1641012c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:16.815 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.834+0000 7ff16b9ff700 1 -- 172.21.15.17:0/244612129 >> 172.21.15.17:0/244612129 conn(0x7ff1640fb200 msgr2=0x7ff1640fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:16.815 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.834+0000 7ff16b9ff700 1 -- 172.21.15.17:0/244612129 shutdown_connections 2023-07-14T12:01:16.815 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.834+0000 7ff16b9ff700 1 -- 172.21.15.17:0/244612129 wait complete. 2023-07-14T12:01:16.817 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.835+0000 7ff16b9ff700 1 Processor -- start 2023-07-14T12:01:16.817 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.836+0000 7ff16b9ff700 1 -- start start 2023-07-14T12:01:16.817 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.836+0000 7ff16b9ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1640ffbc0 0x7ff1640742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:16.817 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.836+0000 7ff16b9ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff164100e50 0x7ff164072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:16.818 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.836+0000 7ff16b9ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff164101800 0x7ff164072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:16.818 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.836+0000 7ff16b9ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1640735e0 con 0x7ff1640ffbc0 2023-07-14T12:01:16.818 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.836+0000 7ff16b9ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff164073750 con 0x7ff164101800 2023-07-14T12:01:16.818 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.836+0000 7ff16b9ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16419f920 con 0x7ff164100e50 2023-07-14T12:01:16.818 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.836+0000 7ff16979b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1640ffbc0 0x7ff1640742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:16.818 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.837+0000 7ff16979b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1640ffbc0 0x7ff1640742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53536/0 (socket says 172.21.15.17:53536) 2023-07-14T12:01:16.818 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.837+0000 7ff168f9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff164100e50 0x7ff164072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:16.819 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.837+0000 7ff16979b700 1 -- 172.21.15.17:0/2221888350 learned_addr learned my addr 172.21.15.17:0/2221888350 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:16.819 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.837+0000 7ff169f9c700 1 --2- 172.21.15.17:0/2221888350 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff164101800 0x7ff164072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:16.819 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.837+0000 7ff16979b700 1 -- 172.21.15.17:0/2221888350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff164100e50 msgr2=0x7ff164072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:16.819 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.837+0000 7ff16979b700 1 --2- 172.21.15.17:0/2221888350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff164100e50 0x7ff164072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:16.819 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.837+0000 7ff16979b700 1 -- 172.21.15.17:0/2221888350 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff164101800 msgr2=0x7ff164072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:16.819 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.837+0000 7ff16979b700 1 --2- 172.21.15.17:0/2221888350 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff164101800 0x7ff164072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:16.819 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.837+0000 7ff16979b700 1 -- 172.21.15.17:0/2221888350 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff15c009c40 con 0x7ff1640ffbc0 2023-07-14T12:01:16.820 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.837+0000 7ff16979b700 1 --2- 172.21.15.17:0/2221888350 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1640ffbc0 0x7ff1640742c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff16000e9a0 tx=0x7ff16000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:16.820 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.838+0000 7ff15a7fc700 1 -- 172.21.15.17:0/2221888350 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff160009790 con 0x7ff1640ffbc0 2023-07-14T12:01:16.821 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.838+0000 7ff16b9ff700 1 -- 172.21.15.17:0/2221888350 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1540095f0 con 0x7ff1640ffbc0 2023-07-14T12:01:16.821 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.838+0000 7ff15a7fc700 1 -- 172.21.15.17:0/2221888350 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff16000ce80 con 0x7ff1640ffbc0 2023-07-14T12:01:16.821 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.838+0000 7ff15a7fc700 1 -- 172.21.15.17:0/2221888350 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff160018680 con 0x7ff1640ffbc0 2023-07-14T12:01:16.821 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.839+0000 7ff16b9ff700 1 -- 172.21.15.17:0/2221888350 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff16419ffd0 con 0x7ff1640ffbc0 2023-07-14T12:01:16.821 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.840+0000 7ff16b9ff700 1 -- 172.21.15.17:0/2221888350 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff16410b3e0 con 0x7ff1640ffbc0 2023-07-14T12:01:16.822 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.841+0000 7ff15a7fc700 1 -- 172.21.15.17:0/2221888350 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff160010430 con 0x7ff1640ffbc0 2023-07-14T12:01:16.823 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.843+0000 7ff15a7fc700 1 --2- 172.21.15.17:0/2221888350 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7ff1500a1150 0x7ff1500a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:16.823 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.843+0000 7ff15a7fc700 1 -- 172.21.15.17:0/2221888350 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7ff160014070 con 0x7ff1640ffbc0 2023-07-14T12:01:16.823 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.843+0000 7ff168f9a700 1 --2- 172.21.15.17:0/2221888350 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7ff1500a1150 0x7ff1500a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:16.824 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.844+0000 7ff168f9a700 1 --2- 172.21.15.17:0/2221888350 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7ff1500a1150 0x7ff1500a3600 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff15400b2e0 tx=0x7ff154005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:16.826 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:16.846+0000 7ff15a7fc700 1 -- 172.21.15.17:0/2221888350 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff160051790 con 0x7ff1640ffbc0 2023-07-14T12:01:16.894 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:16 smithi017 ceph-mon[126339]: pgmap v327: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s; 12 B/s, 0 objects/s recovering 2023-07-14T12:01:16.894 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:16 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/2660906681' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T12:01:16.894 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:16 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/1363131501' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T12:01:17.036 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.056+0000 7ff16b9ff700 1 -- 172.21.15.17:0/2221888350 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff164106b10 con 0x7ff1640ffbc0 2023-07-14T12:01:17.037 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.057+0000 7ff15a7fc700 1 -- 172.21.15.17:0/2221888350 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff160096240 con 0x7ff1640ffbc0 2023-07-14T12:01:17.037 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2023-07-14T12:01:17.039 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.059+0000 7ff16b9ff700 1 -- 172.21.15.17:0/2221888350 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7ff1500a1150 msgr2=0x7ff1500a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:17.039 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.059+0000 7ff16b9ff700 1 --2- 172.21.15.17:0/2221888350 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7ff1500a1150 0x7ff1500a3600 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff15400b2e0 tx=0x7ff154005fb0 comp rx=0 tx=0).stop 2023-07-14T12:01:17.040 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.059+0000 7ff16b9ff700 1 -- 172.21.15.17:0/2221888350 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1640ffbc0 msgr2=0x7ff1640742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:17.040 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.059+0000 7ff16b9ff700 1 --2- 172.21.15.17:0/2221888350 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1640ffbc0 0x7ff1640742c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff16000e9a0 tx=0x7ff16000ecb0 comp rx=0 tx=0).stop 2023-07-14T12:01:17.040 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.059+0000 7ff16b9ff700 1 -- 172.21.15.17:0/2221888350 shutdown_connections 2023-07-14T12:01:17.040 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.059+0000 7ff16b9ff700 1 --2- 172.21.15.17:0/2221888350 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7ff1500a1150 0x7ff1500a3600 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:17.040 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.059+0000 7ff16b9ff700 1 --2- 172.21.15.17:0/2221888350 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff164101800 0x7ff164072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:17.040 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.059+0000 7ff16b9ff700 1 --2- 172.21.15.17:0/2221888350 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1640ffbc0 0x7ff1640742c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:17.040 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.059+0000 7ff16b9ff700 1 --2- 172.21.15.17:0/2221888350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff164100e50 0x7ff164072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:17.041 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.059+0000 7ff16b9ff700 1 -- 172.21.15.17:0/2221888350 >> 172.21.15.17:0/2221888350 conn(0x7ff1640fb200 msgr2=0x7ff1640fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:17.041 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.059+0000 7ff16b9ff700 1 -- 172.21.15.17:0/2221888350 shutdown_connections 2023-07-14T12:01:17.041 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.059+0000 7ff16b9ff700 1 -- 172.21.15.17:0/2221888350 wait complete. 2023-07-14T12:01:17.058 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T12:01:17.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:16 smithi093 ceph-mon[128922]: pgmap v327: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s; 12 B/s, 0 objects/s recovering 2023-07-14T12:01:17.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:16 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/2660906681' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T12:01:17.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:16 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/1363131501' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T12:01:17.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:16 smithi097 ceph-mon[129313]: pgmap v327: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s; 12 B/s, 0 objects/s recovering 2023-07-14T12:01:17.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:16 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/2660906681' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T12:01:17.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:16 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/1363131501' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T12:01:17.262 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.282+0000 7f193dfff700 1 -- 172.21.15.17:0/3252602212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19380ffba0 msgr2=0x7f19380fffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:17.263 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.282+0000 7f193dfff700 1 --2- 172.21.15.17:0/3252602212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19380ffba0 0x7f19380fffb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1928009990 tx=0x7f1928009ca0 comp rx=0 tx=0).stop 2023-07-14T12:01:17.263 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.282+0000 7f193dfff700 1 -- 172.21.15.17:0/3252602212 shutdown_connections 2023-07-14T12:01:17.263 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.282+0000 7f193dfff700 1 --2- 172.21.15.17:0/3252602212 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19381017e0 0x7f1938108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:17.263 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.282+0000 7f193dfff700 1 --2- 172.21.15.17:0/3252602212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19380ffba0 0x7f19380fffb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:17.263 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.282+0000 7f193dfff700 1 --2- 172.21.15.17:0/3252602212 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1938100e30 0x7f19381012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:17.264 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.282+0000 7f193dfff700 1 -- 172.21.15.17:0/3252602212 >> 172.21.15.17:0/3252602212 conn(0x7f19380fb1c0 msgr2=0x7f19380fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:17.264 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.282+0000 7f193dfff700 1 -- 172.21.15.17:0/3252602212 shutdown_connections 2023-07-14T12:01:17.264 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.282+0000 7f193dfff700 1 -- 172.21.15.17:0/3252602212 wait complete. 2023-07-14T12:01:17.266 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.284+0000 7f193dfff700 1 Processor -- start 2023-07-14T12:01:17.266 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.284+0000 7f193dfff700 1 -- start start 2023-07-14T12:01:17.266 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.284+0000 7f193dfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19380ffba0 0x7f19381958e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:17.266 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.284+0000 7f193dfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1938100e30 0x7f1938195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:17.266 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.284+0000 7f193dfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f19381017e0 0x7f193819ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:17.267 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.284+0000 7f193dfff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193819d460 con 0x7f19380ffba0 2023-07-14T12:01:17.267 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.284+0000 7f193dfff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193819d5a0 con 0x7f19381017e0 2023-07-14T12:01:17.267 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.284+0000 7f193dfff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193819d730 con 0x7f1938100e30 2023-07-14T12:01:17.267 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.285+0000 7f19377fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19380ffba0 0x7f19381958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:17.267 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.285+0000 7f19377fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19380ffba0 0x7f19381958e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53560/0 (socket says 172.21.15.17:53560) 2023-07-14T12:01:17.267 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.285+0000 7f19377fe700 1 -- 172.21.15.17:0/1797241601 learned_addr learned my addr 172.21.15.17:0/1797241601 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:17.268 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.285+0000 7f1936ffd700 1 --2- 172.21.15.17:0/1797241601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1938100e30 0x7f1938195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:17.268 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.285+0000 7f1937fff700 1 --2- 172.21.15.17:0/1797241601 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f19381017e0 0x7f193819ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:17.268 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.285+0000 7f19377fe700 1 -- 172.21.15.17:0/1797241601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1938100e30 msgr2=0x7f1938195e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:17.268 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.285+0000 7f19377fe700 1 --2- 172.21.15.17:0/1797241601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1938100e30 0x7f1938195e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:17.268 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.285+0000 7f19377fe700 1 -- 172.21.15.17:0/1797241601 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f19381017e0 msgr2=0x7f193819ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:17.268 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.285+0000 7f19377fe700 1 --2- 172.21.15.17:0/1797241601 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f19381017e0 0x7f193819ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:17.269 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.285+0000 7f19377fe700 1 -- 172.21.15.17:0/1797241601 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19280095f0 con 0x7f19380ffba0 2023-07-14T12:01:17.269 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.285+0000 7f19377fe700 1 --2- 172.21.15.17:0/1797241601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19380ffba0 0x7f19381958e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f192800b680 tx=0x7f1928005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:17.269 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.286+0000 7f1934ff9700 1 -- 172.21.15.17:0/1797241601 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192801d070 con 0x7f19380ffba0 2023-07-14T12:01:17.269 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.286+0000 7f1934ff9700 1 -- 172.21.15.17:0/1797241601 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1928022470 con 0x7f19380ffba0 2023-07-14T12:01:17.269 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.286+0000 7f1934ff9700 1 -- 172.21.15.17:0/1797241601 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1928017650 con 0x7f19380ffba0 2023-07-14T12:01:17.269 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.286+0000 7f193dfff700 1 -- 172.21.15.17:0/1797241601 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f193819d9b0 con 0x7f19380ffba0 2023-07-14T12:01:17.270 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.286+0000 7f193dfff700 1 -- 172.21.15.17:0/1797241601 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f193819dd80 con 0x7f19380ffba0 2023-07-14T12:01:17.270 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.287+0000 7f193dfff700 1 -- 172.21.15.17:0/1797241601 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f193810b3c0 con 0x7f19380ffba0 2023-07-14T12:01:17.270 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.289+0000 7f1934ff9700 1 -- 172.21.15.17:0/1797241601 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f19280225e0 con 0x7f19380ffba0 2023-07-14T12:01:17.271 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.290+0000 7f1934ff9700 1 --2- 172.21.15.17:0/1797241601 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f19200a1150 0x7f19200a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:17.271 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.290+0000 7f1934ff9700 1 -- 172.21.15.17:0/1797241601 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f19280ce1e0 con 0x7f19380ffba0 2023-07-14T12:01:17.271 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.290+0000 7f1936ffd700 1 --2- 172.21.15.17:0/1797241601 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f19200a1150 0x7f19200a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:17.271 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.291+0000 7f1936ffd700 1 --2- 172.21.15.17:0/1797241601 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f19200a1150 0x7f19200a3600 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f192c009c60 tx=0x7f192c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:17.274 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.294+0000 7f1934ff9700 1 -- 172.21.15.17:0/1797241601 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1928097610 con 0x7f19380ffba0 2023-07-14T12:01:17.529 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.549+0000 7f193dfff700 1 -- 172.21.15.17:0/1797241601 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f193810b5b0 con 0x7f19380ffba0 2023-07-14T12:01:17.530 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.550+0000 7f1934ff9700 1 -- 172.21.15.17:0/1797241601 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10869 (secure 0 0 0) 0x7f192809ac30 con 0x7f19380ffba0 2023-07-14T12:01:17.530 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T12:01:17.530 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/55384193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":55384193},{"type":"v1","addr":"172.21.15.17:6835","nonce":55384193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/424956554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":424956554},{"type":"v1","addr":"172.21.15.17:6837","nonce":424956554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T11:59:43.292866+0000","modified":"2023-07-14T12:00:17.489445+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24565,"mds_1":24557,"mds_2":24541,"mds_3":24533,"mds_4":24517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24452":{"gid":24452,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.93:6833/326319712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":326319712},{"type":"v1","addr":"172.21.15.93:6833","nonce":326319712}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24454":{"gid":24454,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6833/878913679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":878913679},{"type":"v1","addr":"172.21.15.97:6833","nonce":878913679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.93:6835/2779420503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":2779420503},{"type":"v1","addr":"172.21.15.93:6835","nonce":2779420503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/2740799393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2740799393},{"type":"v1","addr":"172.21.15.97:6835","nonce":2740799393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.93:6837/1880673911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":1880673911},{"type":"v1","addr":"172.21.15.93:6837","nonce":1880673911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.97:6837/3684187131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3684187131},{"type":"v1","addr":"172.21.15.97:6837","nonce":3684187131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"g","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.17:6839/2976493015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":2976493015},{"type":"v1","addr":"172.21.15.17:6839","nonce":2976493015}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.97:6839/3267815331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3267815331},{"type":"v1","addr":"172.21.15.97:6839","nonce":3267815331}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.93:6839/1281342428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":1281342428},{"type":"v1","addr":"172.21.15.93:6839","nonce":1281342428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6841/3905201143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6840","nonce":3905201143},{"type":"v1","addr":"172.21.15.17:6841","nonce":3905201143}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T12:01:17.532 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.552+0000 7f193dfff700 1 -- 172.21.15.17:0/1797241601 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f19200a1150 msgr2=0x7f19200a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:17.533 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.552+0000 7f193dfff700 1 --2- 172.21.15.17:0/1797241601 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f19200a1150 0x7f19200a3600 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f192c009c60 tx=0x7f192c009260 comp rx=0 tx=0).stop 2023-07-14T12:01:17.533 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.552+0000 7f193dfff700 1 -- 172.21.15.17:0/1797241601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19380ffba0 msgr2=0x7f19381958e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:17.533 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.552+0000 7f193dfff700 1 --2- 172.21.15.17:0/1797241601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19380ffba0 0x7f19381958e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f192800b680 tx=0x7f1928005740 comp rx=0 tx=0).stop 2023-07-14T12:01:17.533 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.552+0000 7f193dfff700 1 -- 172.21.15.17:0/1797241601 shutdown_connections 2023-07-14T12:01:17.533 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.552+0000 7f193dfff700 1 --2- 172.21.15.17:0/1797241601 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f19200a1150 0x7f19200a3600 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:17.533 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.552+0000 7f193dfff700 1 --2- 172.21.15.17:0/1797241601 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f19381017e0 0x7f193819ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:17.534 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.552+0000 7f193dfff700 1 --2- 172.21.15.17:0/1797241601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19380ffba0 0x7f19381958e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:17.534 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.553+0000 7f193dfff700 1 --2- 172.21.15.17:0/1797241601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1938100e30 0x7f1938195e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:17.534 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.553+0000 7f193dfff700 1 -- 172.21.15.17:0/1797241601 >> 172.21.15.17:0/1797241601 conn(0x7f19380fb1c0 msgr2=0x7f19380fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:17.534 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.553+0000 7f193dfff700 1 -- 172.21.15.17:0/1797241601 shutdown_connections 2023-07-14T12:01:17.534 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.553+0000 7f193dfff700 1 -- 172.21.15.17:0/1797241601 wait complete. 2023-07-14T12:01:17.534 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 27 2023-07-14T12:01:17.552 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T12:01:17.757 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.777+0000 7f7f02fff700 1 -- 172.21.15.17:0/2124207179 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7efc100e40 msgr2=0x7f7efc1012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:17.758 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.777+0000 7f7f02fff700 1 --2- 172.21.15.17:0/2124207179 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7efc100e40 0x7f7efc1012b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7ee80098f0 tx=0x7f7ee8009c00 comp rx=0 tx=0).stop 2023-07-14T12:01:17.758 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.777+0000 7f7f02fff700 1 -- 172.21.15.17:0/2124207179 shutdown_connections 2023-07-14T12:01:17.758 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.777+0000 7f7f02fff700 1 --2- 172.21.15.17:0/2124207179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7efc1017f0 0x7f7efc108090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:17.758 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.777+0000 7f7f02fff700 1 --2- 172.21.15.17:0/2124207179 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7efc0ffbb0 0x7f7efc0fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:17.758 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.777+0000 7f7f02fff700 1 --2- 172.21.15.17:0/2124207179 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7efc100e40 0x7f7efc1012b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:17.758 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.777+0000 7f7f02fff700 1 -- 172.21.15.17:0/2124207179 >> 172.21.15.17:0/2124207179 conn(0x7f7efc0fb1d0 msgr2=0x7f7efc0fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:17.758 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.777+0000 7f7f02fff700 1 -- 172.21.15.17:0/2124207179 shutdown_connections 2023-07-14T12:01:17.759 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.777+0000 7f7f02fff700 1 -- 172.21.15.17:0/2124207179 wait complete. 2023-07-14T12:01:17.760 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.778+0000 7f7f02fff700 1 Processor -- start 2023-07-14T12:01:17.760 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.778+0000 7f7f02fff700 1 -- start start 2023-07-14T12:01:17.761 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.779+0000 7f7f02fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7efc0ffbb0 0x7f7efc197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:17.761 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.779+0000 7f7f02fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7efc100e40 0x7f7efc198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:17.761 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.779+0000 7f7f02fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7efc1017f0 0x7f7efc19f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:17.761 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.779+0000 7f7f02fff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc19f6b0 con 0x7f7efc100e40 2023-07-14T12:01:17.761 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.779+0000 7f7f02fff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc19f7f0 con 0x7f7efc1017f0 2023-07-14T12:01:17.761 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.779+0000 7f7f02fff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc19f980 con 0x7f7efc0ffbb0 2023-07-14T12:01:17.761 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.779+0000 7f7efbfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7efc100e40 0x7f7efc198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:17.761 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.779+0000 7f7efbfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7efc100e40 0x7f7efc198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53578/0 (socket says 172.21.15.17:53578) 2023-07-14T12:01:17.762 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.779+0000 7f7efbfff700 1 -- 172.21.15.17:0/829115467 learned_addr learned my addr 172.21.15.17:0/829115467 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:17.762 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.779+0000 7f7f00d9b700 1 --2- 172.21.15.17:0/829115467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7efc0ffbb0 0x7f7efc197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:17.762 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.779+0000 7f7f0159c700 1 --2- 172.21.15.17:0/829115467 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7efc1017f0 0x7f7efc19f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:17.762 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.779+0000 7f7efbfff700 1 -- 172.21.15.17:0/829115467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7efc0ffbb0 msgr2=0x7f7efc197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:17.762 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.779+0000 7f7efbfff700 1 --2- 172.21.15.17:0/829115467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7efc0ffbb0 0x7f7efc197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:17.762 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.779+0000 7f7efbfff700 1 -- 172.21.15.17:0/829115467 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7efc1017f0 msgr2=0x7f7efc19f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:17.762 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.779+0000 7f7efbfff700 1 --2- 172.21.15.17:0/829115467 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7efc1017f0 0x7f7efc19f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:17.763 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.779+0000 7f7efbfff700 1 -- 172.21.15.17:0/829115467 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee8009520 con 0x7f7efc100e40 2023-07-14T12:01:17.763 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.780+0000 7f7efbfff700 1 --2- 172.21.15.17:0/829115467 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7efc100e40 0x7f7efc198070 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8009fd0 tx=0x7f7ee800b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:17.763 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.780+0000 7f7ef9ffb700 1 -- 172.21.15.17:0/829115467 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee801c070 con 0x7f7efc100e40 2023-07-14T12:01:17.763 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.780+0000 7f7ef9ffb700 1 -- 172.21.15.17:0/829115467 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ee800f7d0 con 0x7f7efc100e40 2023-07-14T12:01:17.763 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.780+0000 7f7ef9ffb700 1 -- 172.21.15.17:0/829115467 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8004ab0 con 0x7f7efc100e40 2023-07-14T12:01:17.763 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.780+0000 7f7f02fff700 1 -- 172.21.15.17:0/829115467 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7efc19fc00 con 0x7f7efc100e40 2023-07-14T12:01:17.763 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.780+0000 7f7f02fff700 1 -- 172.21.15.17:0/829115467 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7efc19ffd0 con 0x7f7efc100e40 2023-07-14T12:01:17.764 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.781+0000 7f7f02fff700 1 -- 172.21.15.17:0/829115467 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7efc192290 con 0x7f7efc100e40 2023-07-14T12:01:17.764 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.783+0000 7f7ef9ffb700 1 -- 172.21.15.17:0/829115467 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7ee8020070 con 0x7f7efc100e40 2023-07-14T12:01:17.765 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.785+0000 7f7ef9ffb700 1 --2- 172.21.15.17:0/829115467 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f7eec0a1150 0x7f7eec0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:17.765 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.785+0000 7f7ef9ffb700 1 -- 172.21.15.17:0/829115467 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f7ee80cd8e0 con 0x7f7efc100e40 2023-07-14T12:01:17.765 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.785+0000 7f7f00d9b700 1 --2- 172.21.15.17:0/829115467 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f7eec0a1150 0x7f7eec0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:17.766 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.785+0000 7f7f00d9b700 1 --2- 172.21.15.17:0/829115467 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f7eec0a1150 0x7f7eec0a3600 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0005fd0 tx=0x7f7ef0009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:17.768 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.788+0000 7f7ef9ffb700 1 -- 172.21.15.17:0/829115467 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7ee805c790 con 0x7f7efc100e40 2023-07-14T12:01:17.978 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.998+0000 7f7f02fff700 1 -- 172.21.15.17:0/829115467 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7efc02c850 con 0x7f7efc100e40 2023-07-14T12:01:17.979 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:17.999+0000 7f7ef9ffb700 1 -- 172.21.15.17:0/829115467 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24486 (secure 0 0 0) 0x7f7ee8096d10 con 0x7f7efc100e40 2023-07-14T12:01:17.980 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T12:01:17.980 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":92,"fsid":"b3c386cc-223c-11ee-9b33-001a4aab830c","created":"2023-07-14T11:51:59.992960+0000","modified":"2023-07-14T12:01:07.806062+0000","last_up_change":"2023-07-14T11:58:55.037769+0000","last_in_change":"2023-07-14T11:58:37.114870+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:55:28.237002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:59:41.560502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T11:59:42.725461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:59:46.267029+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"57003438-a83c-4c8e-a029-56414740895c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6803","nonce":3588366384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6805","nonce":3588366384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6809","nonce":3588366384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6807","nonce":3588366384}]},"public_addr":"172.21.15.17:6803/3588366384","cluster_addr":"172.21.15.17:6805/3588366384","heartbeat_back_addr":"172.21.15.17:6809/3588366384","heartbeat_front_addr":"172.21.15.17:6807/3588366384","state":["exists","up"]},{"osd":1,"uuid":"f56b6337-d8a9-4411-b9d8-b6863dc98556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6801","nonce":3147203064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6803","nonce":3147203064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6807","nonce":3147203064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6805","nonce":3147203064}]},"public_addr":"172.21.15.93:6801/3147203064","cluster_addr":"172.21.15.93:6803/3147203064","heartbeat_back_addr":"172.21.15.93:6807/3147203064","heartbeat_front_addr":"172.21.15.93:6805/3147203064","state":["exists","up"]},{"osd":2,"uuid":"fc34bb7f-551f-46e1-99c0-59924d6c1346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6801","nonce":1877269815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6803","nonce":1877269815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6807","nonce":1877269815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6805","nonce":1877269815}]},"public_addr":"172.21.15.97:6801/1877269815","cluster_addr":"172.21.15.97:6803/1877269815","heartbeat_back_addr":"172.21.15.97:6807/1877269815","heartbeat_front_addr":"172.21.15.97:6805/1877269815","state":["exists","up"]},{"osd":3,"uuid":"37a1583c-4fb8-412a-a0c8-f83b1ff26a91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6811","nonce":2784619821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6813","nonce":2784619821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6817","nonce":2784619821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6815","nonce":2784619821}]},"public_addr":"172.21.15.17:6811/2784619821","cluster_addr":"172.21.15.17:6813/2784619821","heartbeat_back_addr":"172.21.15.17:6817/2784619821","heartbeat_front_addr":"172.21.15.17:6815/2784619821","state":["exists","up"]},{"osd":4,"uuid":"f4b9f377-d519-4b14-ac4a-f1debaeed9f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6809","nonce":3151801173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6811","nonce":3151801173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6815","nonce":3151801173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6813","nonce":3151801173}]},"public_addr":"172.21.15.93:6809/3151801173","cluster_addr":"172.21.15.93:6811/3151801173","heartbeat_back_addr":"172.21.15.93:6815/3151801173","heartbeat_front_addr":"172.21.15.93:6813/3151801173","state":["exists","up"]},{"osd":5,"uuid":"2d2af354-0535-4ef9-b166-3ef219498db8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6809","nonce":2184413924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6811","nonce":2184413924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6815","nonce":2184413924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6813","nonce":2184413924}]},"public_addr":"172.21.15.97:6809/2184413924","cluster_addr":"172.21.15.97:6811/2184413924","heartbeat_back_addr":"172.21.15.97:6815/2184413924","heartbeat_front_addr":"172.21.15.97:6813/2184413924","state":["exists","up"]},{"osd":6,"uuid":"9cdf0c4b-62dc-4168-864e-a9d1650a9ea8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6819","nonce":50486581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6821","nonce":50486581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6825","nonce":50486581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6823","nonce":50486581}]},"public_addr":"172.21.15.17:6819/50486581","cluster_addr":"172.21.15.17:6821/50486581","heartbeat_back_addr":"172.21.15.17:6825/50486581","heartbeat_front_addr":"172.21.15.17:6823/50486581","state":["exists","up"]},{"osd":7,"uuid":"09f60823-5491-45cd-9324-6ae0d56ce262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6817","nonce":2596912392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6819","nonce":2596912392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6823","nonce":2596912392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6821","nonce":2596912392}]},"public_addr":"172.21.15.93:6817/2596912392","cluster_addr":"172.21.15.93:6819/2596912392","heartbeat_back_addr":"172.21.15.93:6823/2596912392","heartbeat_front_addr":"172.21.15.93:6821/2596912392","state":["exists","up"]},{"osd":8,"uuid":"8c5771c9-718e-4313-a2b6-5e1c37c53f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6817","nonce":1965492859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6819","nonce":1965492859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6823","nonce":1965492859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6821","nonce":1965492859}]},"public_addr":"172.21.15.97:6817/1965492859","cluster_addr":"172.21.15.97:6819/1965492859","heartbeat_back_addr":"172.21.15.97:6823/1965492859","heartbeat_front_addr":"172.21.15.97:6821/1965492859","state":["exists","up"]},{"osd":9,"uuid":"1197b8a3-c09f-4c57-aba1-9e988a1620b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6827","nonce":2512645700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6829","nonce":2512645700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6833","nonce":2512645700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6831","nonce":2512645700}]},"public_addr":"172.21.15.17:6827/2512645700","cluster_addr":"172.21.15.17:6829/2512645700","heartbeat_back_addr":"172.21.15.17:6833/2512645700","heartbeat_front_addr":"172.21.15.17:6831/2512645700","state":["exists","up"]},{"osd":10,"uuid":"f99918c2-cdd8-4a7a-be2e-d1d6e0f828a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6825","nonce":264429558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6827","nonce":264429558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6831","nonce":264429558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6829","nonce":264429558}]},"public_addr":"172.21.15.93:6825/264429558","cluster_addr":"172.21.15.93:6827/264429558","heartbeat_back_addr":"172.21.15.93:6831/264429558","heartbeat_front_addr":"172.21.15.93:6829/264429558","state":["exists","up"]},{"osd":11,"uuid":"6bbb5b79-8d4d-441f-9583-203df6c78cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6825","nonce":3389913754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6827","nonce":3389913754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6831","nonce":3389913754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6829","nonce":3389913754}]},"public_addr":"172.21.15.97:6825/3389913754","cluster_addr":"172.21.15.97:6827/3389913754","heartbeat_back_addr":"172.21.15.97:6831/3389913754","heartbeat_front_addr":"172.21.15.97:6829/3389913754","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:54:41.079618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:03.207899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:24.136961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:50.392609+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:56:12.136237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:56:33.212314+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:00.243646+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:22.059234+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:43.232399+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:10.043896+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:31.910780+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:53.158983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.9","mappings":[{"from":5,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/2084033276":"2023-07-15T11:53:05.639597+0000","172.21.15.17:6801/1059284716":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/3651785636":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/1373538024":"2023-07-15T11:52:37.887946+0000","172.21.15.17:0/3560395020":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6800/1059284716":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/4262912861":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/2258375202":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6800/2804227742":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6801/2804227742":"2023-07-15T11:52:37.887946+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T12:01:17.982 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.002+0000 7f7f02fff700 1 -- 172.21.15.17:0/829115467 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f7eec0a1150 msgr2=0x7f7eec0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:17.982 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.002+0000 7f7f02fff700 1 --2- 172.21.15.17:0/829115467 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f7eec0a1150 0x7f7eec0a3600 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0005fd0 tx=0x7f7ef0009310 comp rx=0 tx=0).stop 2023-07-14T12:01:17.983 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.002+0000 7f7f02fff700 1 -- 172.21.15.17:0/829115467 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7efc100e40 msgr2=0x7f7efc198070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:17.983 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.002+0000 7f7f02fff700 1 --2- 172.21.15.17:0/829115467 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7efc100e40 0x7f7efc198070 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8009fd0 tx=0x7f7ee800b790 comp rx=0 tx=0).stop 2023-07-14T12:01:17.983 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.002+0000 7f7f02fff700 1 -- 172.21.15.17:0/829115467 shutdown_connections 2023-07-14T12:01:17.983 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.002+0000 7f7f02fff700 1 --2- 172.21.15.17:0/829115467 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f7eec0a1150 0x7f7eec0a3600 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:17.983 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.002+0000 7f7f02fff700 1 --2- 172.21.15.17:0/829115467 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7efc1017f0 0x7f7efc19f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:17.983 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.002+0000 7f7f02fff700 1 --2- 172.21.15.17:0/829115467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7efc0ffbb0 0x7f7efc197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:17.983 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.002+0000 7f7f02fff700 1 --2- 172.21.15.17:0/829115467 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7efc100e40 0x7f7efc198070 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:17.984 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.002+0000 7f7f02fff700 1 -- 172.21.15.17:0/829115467 >> 172.21.15.17:0/829115467 conn(0x7f7efc0fb1d0 msgr2=0x7f7efc106740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:17.984 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.002+0000 7f7f02fff700 1 -- 172.21.15.17:0/829115467 shutdown_connections 2023-07-14T12:01:17.984 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.002+0000 7f7f02fff700 1 -- 172.21.15.17:0/829115467 wait complete. 2023-07-14T12:01:18.001 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T12:01:18.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:17 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/2221888350' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T12:01:18.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:17 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/1797241601' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T12:01:18.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:17 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/2221888350' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T12:01:18.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:17 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/1797241601' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T12:01:18.143 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:17 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/2221888350' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-14T12:01:18.143 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:17 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/1797241601' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T12:01:18.206 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.225+0000 7f43e31ff700 1 -- 172.21.15.17:0/2957221050 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43dc100e30 msgr2=0x7f43dc1012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:18.206 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.225+0000 7f43e31ff700 1 --2- 172.21.15.17:0/2957221050 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43dc100e30 0x7f43dc1012a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f43c80098f0 tx=0x7f43c8009c00 comp rx=0 tx=0).stop 2023-07-14T12:01:18.207 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.225+0000 7f43e31ff700 1 -- 172.21.15.17:0/2957221050 shutdown_connections 2023-07-14T12:01:18.207 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.225+0000 7f43e31ff700 1 --2- 172.21.15.17:0/2957221050 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f43dc0ffba0 0x7f43dc0fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.207 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.225+0000 7f43e31ff700 1 --2- 172.21.15.17:0/2957221050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43dc1017e0 0x7f43dc108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.207 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.225+0000 7f43e31ff700 1 --2- 172.21.15.17:0/2957221050 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43dc100e30 0x7f43dc1012a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.207 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.225+0000 7f43e31ff700 1 -- 172.21.15.17:0/2957221050 >> 172.21.15.17:0/2957221050 conn(0x7f43dc0fb1c0 msgr2=0x7f43dc0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:18.207 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.225+0000 7f43e31ff700 1 -- 172.21.15.17:0/2957221050 shutdown_connections 2023-07-14T12:01:18.207 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.225+0000 7f43e31ff700 1 -- 172.21.15.17:0/2957221050 wait complete. 2023-07-14T12:01:18.210 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.226+0000 7f43e31ff700 1 Processor -- start 2023-07-14T12:01:18.210 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.227+0000 7f43e31ff700 1 -- start start 2023-07-14T12:01:18.210 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.227+0000 7f43e31ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f43dc0ffba0 0x7f43dc197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:18.210 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.227+0000 7f43e31ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43dc100e30 0x7f43dc198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:18.210 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.227+0000 7f43e31ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43dc1017e0 0x7f43dc19f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:18.211 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.227+0000 7f43e31ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43dc19f6d0 con 0x7f43dc1017e0 2023-07-14T12:01:18.211 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.227+0000 7f43e31ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43dc19f810 con 0x7f43dc0ffba0 2023-07-14T12:01:18.211 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.227+0000 7f43e31ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43dc19f9a0 con 0x7f43dc100e30 2023-07-14T12:01:18.211 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.227+0000 7f43e179c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43dc1017e0 0x7f43dc19f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:18.211 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.227+0000 7f43e179c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43dc1017e0 0x7f43dc19f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53590/0 (socket says 172.21.15.17:53590) 2023-07-14T12:01:18.211 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.227+0000 7f43e179c700 1 -- 172.21.15.17:0/4290558975 learned_addr learned my addr 172.21.15.17:0/4290558975 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:18.211 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.227+0000 7f43dbfff700 1 --2- 172.21.15.17:0/4290558975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43dc100e30 0x7f43dc198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:18.212 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.227+0000 7f43e0f9b700 1 --2- 172.21.15.17:0/4290558975 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f43dc0ffba0 0x7f43dc197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:18.212 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.228+0000 7f43e179c700 1 -- 172.21.15.17:0/4290558975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43dc100e30 msgr2=0x7f43dc198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:18.212 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.228+0000 7f43e179c700 1 --2- 172.21.15.17:0/4290558975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43dc100e30 0x7f43dc198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.212 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.228+0000 7f43e179c700 1 -- 172.21.15.17:0/4290558975 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f43dc0ffba0 msgr2=0x7f43dc197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:18.212 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.228+0000 7f43e179c700 1 --2- 172.21.15.17:0/4290558975 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f43dc0ffba0 0x7f43dc197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.212 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.228+0000 7f43e179c700 1 -- 172.21.15.17:0/4290558975 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d00095f0 con 0x7f43dc1017e0 2023-07-14T12:01:18.213 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.228+0000 7f43e179c700 1 --2- 172.21.15.17:0/4290558975 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43dc1017e0 0x7f43dc19f100 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f43d400ef30 tx=0x7f43d400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:18.213 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.228+0000 7f43d9ffb700 1 -- 172.21.15.17:0/4290558975 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d4018070 con 0x7f43dc1017e0 2023-07-14T12:01:18.213 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.228+0000 7f43d9ffb700 1 -- 172.21.15.17:0/4290558975 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43d400f040 con 0x7f43dc1017e0 2023-07-14T12:01:18.213 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.228+0000 7f43d9ffb700 1 -- 172.21.15.17:0/4290558975 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d400cd00 con 0x7f43dc1017e0 2023-07-14T12:01:18.213 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.228+0000 7f43e31ff700 1 -- 172.21.15.17:0/4290558975 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43c8009520 con 0x7f43dc1017e0 2023-07-14T12:01:18.213 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.228+0000 7f43e31ff700 1 -- 172.21.15.17:0/4290558975 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43dc19ff50 con 0x7f43dc1017e0 2023-07-14T12:01:18.214 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.230+0000 7f43e31ff700 1 -- 172.21.15.17:0/4290558975 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43dc10b3c0 con 0x7f43dc1017e0 2023-07-14T12:01:18.215 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.232+0000 7f43d9ffb700 1 -- 172.21.15.17:0/4290558975 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f43d4003a00 con 0x7f43dc1017e0 2023-07-14T12:01:18.215 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.233+0000 7f43d9ffb700 1 --2- 172.21.15.17:0/4290558975 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f43cc0a11d0 0x7f43cc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:18.215 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.233+0000 7f43d9ffb700 1 -- 172.21.15.17:0/4290558975 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f43d40cc6c0 con 0x7f43dc1017e0 2023-07-14T12:01:18.215 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.233+0000 7f43e0f9b700 1 --2- 172.21.15.17:0/4290558975 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f43cc0a11d0 0x7f43cc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:18.215 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.234+0000 7f43e0f9b700 1 --2- 172.21.15.17:0/4290558975 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f43cc0a11d0 0x7f43cc0a3680 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f43d0005fd0 tx=0x7f43d0009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:18.217 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.237+0000 7f43d9ffb700 1 -- 172.21.15.17:0/4290558975 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f43d4095b50 con 0x7f43dc1017e0 2023-07-14T12:01:18.470 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.490+0000 7f43e31ff700 1 -- 172.21.15.17:0/4290558975 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f43dc10b5b0 con 0x7f43dc1017e0 2023-07-14T12:01:18.471 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.491+0000 7f43d9ffb700 1 -- 172.21.15.17:0/4290558975 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10869 (secure 0 0 0) 0x7f43d4099170 con 0x7f43dc1017e0 2023-07-14T12:01:18.471 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T12:01:18.471 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/55384193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":55384193},{"type":"v1","addr":"172.21.15.17:6835","nonce":55384193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/424956554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":424956554},{"type":"v1","addr":"172.21.15.17:6837","nonce":424956554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T11:59:43.292866+0000","modified":"2023-07-14T12:00:17.489445+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24565,"mds_1":24557,"mds_2":24541,"mds_3":24533,"mds_4":24517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24452":{"gid":24452,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.93:6833/326319712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":326319712},{"type":"v1","addr":"172.21.15.93:6833","nonce":326319712}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24454":{"gid":24454,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6833/878913679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":878913679},{"type":"v1","addr":"172.21.15.97:6833","nonce":878913679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.93:6835/2779420503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":2779420503},{"type":"v1","addr":"172.21.15.93:6835","nonce":2779420503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/2740799393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2740799393},{"type":"v1","addr":"172.21.15.97:6835","nonce":2740799393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.93:6837/1880673911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":1880673911},{"type":"v1","addr":"172.21.15.93:6837","nonce":1880673911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.97:6837/3684187131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3684187131},{"type":"v1","addr":"172.21.15.97:6837","nonce":3684187131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"g","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.17:6839/2976493015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":2976493015},{"type":"v1","addr":"172.21.15.17:6839","nonce":2976493015}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.97:6839/3267815331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3267815331},{"type":"v1","addr":"172.21.15.97:6839","nonce":3267815331}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.93:6839/1281342428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":1281342428},{"type":"v1","addr":"172.21.15.93:6839","nonce":1281342428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6841/3905201143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6840","nonce":3905201143},{"type":"v1","addr":"172.21.15.17:6841","nonce":3905201143}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T12:01:18.473 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.493+0000 7f43e31ff700 1 -- 172.21.15.17:0/4290558975 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f43cc0a11d0 msgr2=0x7f43cc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:18.473 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.493+0000 7f43e31ff700 1 --2- 172.21.15.17:0/4290558975 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f43cc0a11d0 0x7f43cc0a3680 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f43d0005fd0 tx=0x7f43d0009310 comp rx=0 tx=0).stop 2023-07-14T12:01:18.473 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.493+0000 7f43e31ff700 1 -- 172.21.15.17:0/4290558975 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43dc1017e0 msgr2=0x7f43dc19f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:18.474 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.493+0000 7f43e31ff700 1 --2- 172.21.15.17:0/4290558975 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43dc1017e0 0x7f43dc19f100 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f43d400ef30 tx=0x7f43d400c3a0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.474 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.493+0000 7f43e31ff700 1 -- 172.21.15.17:0/4290558975 shutdown_connections 2023-07-14T12:01:18.474 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.493+0000 7f43e31ff700 1 --2- 172.21.15.17:0/4290558975 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f43cc0a11d0 0x7f43cc0a3680 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.474 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.493+0000 7f43e31ff700 1 --2- 172.21.15.17:0/4290558975 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f43dc0ffba0 0x7f43dc197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.474 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.493+0000 7f43e31ff700 1 --2- 172.21.15.17:0/4290558975 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43dc1017e0 0x7f43dc19f100 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.474 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.493+0000 7f43e31ff700 1 --2- 172.21.15.17:0/4290558975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43dc100e30 0x7f43dc198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.474 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.493+0000 7f43e31ff700 1 -- 172.21.15.17:0/4290558975 >> 172.21.15.17:0/4290558975 conn(0x7f43dc0fb1c0 msgr2=0x7f43dc0fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:18.475 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.493+0000 7f43e31ff700 1 -- 172.21.15.17:0/4290558975 shutdown_connections 2023-07-14T12:01:18.475 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.493+0000 7f43e31ff700 1 -- 172.21.15.17:0/4290558975 wait complete. 2023-07-14T12:01:18.475 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 27 2023-07-14T12:01:18.493 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-07-14T11:59:43.292866+0000', 'modified': '2023-07-14T12:00:17.489445+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24565, 'mds_1': 24557, 'mds_2': 24541, 'mds_3': 24533, 'mds_4': 24517}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24452': {'gid': 24452, 'name': 'k', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.93:6833/326319712', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.93:6832', 'nonce': 326319712}, {'type': 'v1', 'addr': '172.21.15.93:6833', 'nonce': 326319712}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24454': {'gid': 24454, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.97:6833/878913679', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6832', 'nonce': 878913679}, {'type': 'v1', 'addr': '172.21.15.97:6833', 'nonce': 878913679}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24482': {'gid': 24482, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.93:6835/2779420503', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.93:6834', 'nonce': 2779420503}, {'type': 'v1', 'addr': '172.21.15.93:6835', 'nonce': 2779420503}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24505': {'gid': 24505, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.97:6835/2740799393', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6834', 'nonce': 2740799393}, {'type': 'v1', 'addr': '172.21.15.97:6835', 'nonce': 2740799393}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24509': {'gid': 24509, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.93:6837/1880673911', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.93:6836', 'nonce': 1880673911}, {'type': 'v1', 'addr': '172.21.15.93:6837', 'nonce': 1880673911}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24517': {'gid': 24517, 'name': 'c', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.97:6837/3684187131', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6836', 'nonce': 3684187131}, {'type': 'v1', 'addr': '172.21.15.97:6837', 'nonce': 3684187131}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24533': {'gid': 24533, 'name': 'g', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.17:6839/2976493015', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6838', 'nonce': 2976493015}, {'type': 'v1', 'addr': '172.21.15.17:6839', 'nonce': 2976493015}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24541': {'gid': 24541, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.97:6839/3267815331', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6838', 'nonce': 3267815331}, {'type': 'v1', 'addr': '172.21.15.97:6839', 'nonce': 3267815331}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24557': {'gid': 24557, 'name': 'b', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.93:6839/1281342428', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.93:6838', 'nonce': 1281342428}, {'type': 'v1', 'addr': '172.21.15.93:6839', 'nonce': 1281342428}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24565': {'gid': 24565, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6841/3905201143', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6840', 'nonce': 3905201143}, {'type': 'v1', 'addr': '172.21.15.17:6841', 'nonce': 3905201143}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-14T12:01:18.494 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-07-14T12:01:18.495 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-07-14T12:01:18.701 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.720+0000 7f51af5ff700 1 -- 172.21.15.17:0/1782498225 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51a8101800 msgr2=0x7f51a81080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:18.701 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.720+0000 7f51af5ff700 1 --2- 172.21.15.17:0/1782498225 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51a8101800 0x7f51a81080a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f51a000b0a0 tx=0x7f51a000b3b0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.701 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.720+0000 7f51af5ff700 1 -- 172.21.15.17:0/1782498225 shutdown_connections 2023-07-14T12:01:18.701 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.720+0000 7f51af5ff700 1 --2- 172.21.15.17:0/1782498225 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51a80ffbc0 0x7f51a80fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.701 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.720+0000 7f51af5ff700 1 --2- 172.21.15.17:0/1782498225 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51a8101800 0x7f51a81080a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.702 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.720+0000 7f51af5ff700 1 --2- 172.21.15.17:0/1782498225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51a8100e50 0x7f51a81012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.702 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.720+0000 7f51af5ff700 1 -- 172.21.15.17:0/1782498225 >> 172.21.15.17:0/1782498225 conn(0x7f51a80fb200 msgr2=0x7f51a80fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:18.702 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.720+0000 7f51af5ff700 1 -- 172.21.15.17:0/1782498225 shutdown_connections 2023-07-14T12:01:18.702 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.721+0000 7f51af5ff700 1 -- 172.21.15.17:0/1782498225 wait complete. 2023-07-14T12:01:18.703 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.721+0000 7f51af5ff700 1 Processor -- start 2023-07-14T12:01:18.704 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.722+0000 7f51af5ff700 1 -- start start 2023-07-14T12:01:18.704 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.722+0000 7f51af5ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51a80ffbc0 0x7f51a81959e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:18.704 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.722+0000 7f51af5ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51a8100e50 0x7f51a8195f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:18.704 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.722+0000 7f51af5ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51a8101800 0x7f51a819cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:18.704 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.722+0000 7f51af5ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a819d560 con 0x7f51a80ffbc0 2023-07-14T12:01:18.704 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.722+0000 7f51af5ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a819d6a0 con 0x7f51a8100e50 2023-07-14T12:01:18.705 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.722+0000 7f51af5ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a819d830 con 0x7f51a8101800 2023-07-14T12:01:18.705 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.722+0000 7f51ad39b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51a80ffbc0 0x7f51a81959e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:18.705 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.722+0000 7f51ad39b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51a80ffbc0 0x7f51a81959e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53602/0 (socket says 172.21.15.17:53602) 2023-07-14T12:01:18.705 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.722+0000 7f51ad39b700 1 -- 172.21.15.17:0/2270205062 learned_addr learned my addr 172.21.15.17:0/2270205062 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:18.705 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.722+0000 7f51acb9a700 1 --2- 172.21.15.17:0/2270205062 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51a8100e50 0x7f51a8195f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:18.705 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.723+0000 7f51adb9c700 1 --2- 172.21.15.17:0/2270205062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51a8101800 0x7f51a819cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:18.705 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.723+0000 7f51ad39b700 1 -- 172.21.15.17:0/2270205062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51a8101800 msgr2=0x7f51a819cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:18.705 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.723+0000 7f51ad39b700 1 --2- 172.21.15.17:0/2270205062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51a8101800 0x7f51a819cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.706 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.723+0000 7f51ad39b700 1 -- 172.21.15.17:0/2270205062 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51a8100e50 msgr2=0x7f51a8195f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:18.706 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.723+0000 7f51ad39b700 1 --2- 172.21.15.17:0/2270205062 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51a8100e50 0x7f51a8195f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.706 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.723+0000 7f51ad39b700 1 -- 172.21.15.17:0/2270205062 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a4009520 con 0x7f51a80ffbc0 2023-07-14T12:01:18.706 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.723+0000 7f51ad39b700 1 --2- 172.21.15.17:0/2270205062 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51a80ffbc0 0x7f51a81959e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f51a400eb30 tx=0x7f51a400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:18.706 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.723+0000 7f519e7fc700 1 -- 172.21.15.17:0/2270205062 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a4009880 con 0x7f51a80ffbc0 2023-07-14T12:01:18.707 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.723+0000 7f519e7fc700 1 -- 172.21.15.17:0/2270205062 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51a40044e0 con 0x7f51a80ffbc0 2023-07-14T12:01:18.707 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.723+0000 7f51af5ff700 1 -- 172.21.15.17:0/2270205062 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51a0009d00 con 0x7f51a80ffbc0 2023-07-14T12:01:18.707 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.723+0000 7f519e7fc700 1 -- 172.21.15.17:0/2270205062 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a4003d80 con 0x7f51a80ffbc0 2023-07-14T12:01:18.707 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.723+0000 7f51af5ff700 1 -- 172.21.15.17:0/2270205062 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51a819dee0 con 0x7f51a80ffbc0 2023-07-14T12:01:18.711 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.726+0000 7f519e7fc700 1 -- 172.21.15.17:0/2270205062 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f51a4004000 con 0x7f51a80ffbc0 2023-07-14T12:01:18.711 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.728+0000 7f51c2edbb80 1 Processor -- start 2023-07-14T12:01:18.712 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.728+0000 7f519e7fc700 1 --2- 172.21.15.17:0/2270205062 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f51940a11d0 0x7f51940a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:18.712 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.728+0000 7f51c2edbb80 1 -- start start 2023-07-14T12:01:18.712 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.728+0000 7f519e7fc700 1 -- 172.21.15.17:0/2270205062 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f51a4014070 con 0x7f51a80ffbc0 2023-07-14T12:01:18.713 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.728+0000 7f51adb9c700 1 --2- 172.21.15.17:0/2270205062 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f51940a11d0 0x7f51940a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:18.713 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.728+0000 7f51c2edbb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe1504d480 0x55fe1504d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:18.713 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.728+0000 7f51c2edbb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fe1504ddd0 0x55fe15056ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:18.713 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.728+0000 7f51c2edbb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fe15057430 0x55fe150578a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:18.714 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.728+0000 7f51c2edbb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe14f52c60 con 0x55fe1504d480 2023-07-14T12:01:18.714 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.728+0000 7f51c2edbb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe14fa59f0 con 0x55fe15057430 2023-07-14T12:01:18.714 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.728+0000 7f51c2edbb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe14f9ee80 con 0x55fe1504ddd0 2023-07-14T12:01:18.714 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.729+0000 7f51ad39b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe1504d480 0x55fe1504d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:18.715 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.729+0000 7f51ad39b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe1504d480 0x55fe1504d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53614/0 (socket says 172.21.15.17:53614) 2023-07-14T12:01:18.715 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.729+0000 7f51ad39b700 1 -- 172.21.15.17:0/387263883 learned_addr learned my addr 172.21.15.17:0/387263883 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:18.715 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.729+0000 7f51acb9a700 1 --2- 172.21.15.17:0/387263883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fe1504ddd0 0x55fe15056ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:18.715 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.729+0000 7f51adb9c700 1 --2- 172.21.15.17:0/387263883 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fe15057430 0x55fe150578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:18.716 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.729+0000 7f51ad39b700 1 -- 172.21.15.17:0/387263883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fe1504ddd0 msgr2=0x55fe15056ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T12:01:18.716 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.729+0000 7f51ad39b700 1 --2- 172.21.15.17:0/387263883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fe1504ddd0 0x55fe15056ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.716 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.729+0000 7f51adb9c700 1 --2- 172.21.15.17:0/2270205062 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f51940a11d0 0x7f51940a3680 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f51a000b780 tx=0x7f51a00095a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:18.717 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.729+0000 7f51ad39b700 1 -- 172.21.15.17:0/387263883 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fe15057430 msgr2=0x55fe150578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T12:01:18.717 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.729+0000 7f51ad39b700 1 --2- 172.21.15.17:0/387263883 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fe15057430 0x55fe150578a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.717 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.729+0000 7f51ad39b700 1 -- 172.21.15.17:0/387263883 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe14dccd90 con 0x55fe1504d480 2023-07-14T12:01:18.717 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.729+0000 7f51acb9a700 1 --2- 172.21.15.17:0/387263883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fe1504ddd0 0x55fe15056ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T12:01:18.718 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.729+0000 7f51ad39b700 1 --2- 172.21.15.17:0/387263883 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe1504d480 0x55fe1504d890 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f51a40966f0 tx=0x7f51a4003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e0598772aa3a9e7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:18.718 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.730+0000 7f51937fe700 1 -- 172.21.15.17:0/387263883 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a400cbd0 con 0x55fe1504d480 2023-07-14T12:01:18.718 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.730+0000 7f51937fe700 1 -- 172.21.15.17:0/387263883 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51a409a1f0 con 0x55fe1504d480 2023-07-14T12:01:18.718 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.730+0000 7f51937fe700 1 -- 172.21.15.17:0/387263883 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a40983c0 con 0x55fe1504d480 2023-07-14T12:01:18.719 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.730+0000 7f51c2edbb80 1 -- 172.21.15.17:0/387263883 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe1504d480 msgr2=0x55fe1504d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:18.719 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.730+0000 7f51c2edbb80 1 --2- 172.21.15.17:0/387263883 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe1504d480 0x55fe1504d890 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f51a40966f0 tx=0x7f51a4003680 comp rx=0 tx=0).stop 2023-07-14T12:01:18.719 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.730+0000 7f51c2edbb80 1 -- 172.21.15.17:0/387263883 shutdown_connections 2023-07-14T12:01:18.719 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.730+0000 7f51c2edbb80 1 --2- 172.21.15.17:0/387263883 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fe15057430 0x55fe150578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.720 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.730+0000 7f51c2edbb80 1 --2- 172.21.15.17:0/387263883 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe1504d480 0x55fe1504d890 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.720 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.730+0000 7f51c2edbb80 1 --2- 172.21.15.17:0/387263883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fe1504ddd0 0x55fe15056ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.720 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.730+0000 7f51c2edbb80 1 -- 172.21.15.17:0/387263883 >> 172.21.15.17:0/387263883 conn(0x55fe15042d50 msgr2=0x55fe15048180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:18.721 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.731+0000 7f51c2edbb80 1 -- 172.21.15.17:0/387263883 shutdown_connections 2023-07-14T12:01:18.721 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.731+0000 7f51c2edbb80 1 -- 172.21.15.17:0/387263883 wait complete. 2023-07-14T12:01:18.721 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.731+0000 7f51c2edbb80 1 -- start start 2023-07-14T12:01:18.721 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.731+0000 7f51c2edbb80 1 Processor -- start 2023-07-14T12:01:18.726 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.732+0000 7f51c2edbb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe1504d480 0x55fe1505f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:18.727 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.732+0000 7f51c2edbb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fe1504ddd0 0x55fe1505fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:18.727 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.732+0000 7f51c2edbb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fe15057430 0x55fe15066ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:18.727 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.732+0000 7f51c2edbb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe14f3ea70 con 0x55fe1504d480 2023-07-14T12:01:18.727 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.732+0000 7f51c2edbb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe14f3e770 con 0x55fe15057430 2023-07-14T12:01:18.728 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.732+0000 7f51c2edbb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe14e228d0 con 0x55fe1504ddd0 2023-07-14T12:01:18.728 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.732+0000 7f51c2edbb80 10 client.0 _ll_register_callbacks cb 0x55fe14f3c710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T12:01:18.728 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.732+0000 7f51ad39b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe1504d480 0x55fe1505f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:18.728 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.732+0000 7f51ad39b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe1504d480 0x55fe1505f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53620/0 (socket says 172.21.15.17:53620) 2023-07-14T12:01:18.729 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.732+0000 7f51acb9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fe1504ddd0 0x55fe1505fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:18.729 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.732+0000 7f51ad39b700 1 -- 172.21.15.17:0/1143881717 learned_addr learned my addr 172.21.15.17:0/1143881717 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:18.729 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.732+0000 7f51adb9c700 1 --2- 172.21.15.17:0/1143881717 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fe15057430 0x55fe15066ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:18.730 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.732+0000 7f51ad39b700 1 -- 172.21.15.17:0/1143881717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fe1504ddd0 msgr2=0x55fe1505fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T12:01:18.730 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.732+0000 7f51ad39b700 1 --2- 172.21.15.17:0/1143881717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fe1504ddd0 0x55fe1505fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.730 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.732+0000 7f51ad39b700 1 -- 172.21.15.17:0/1143881717 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fe15057430 msgr2=0x55fe15066ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T12:01:18.730 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.732+0000 7f51ad39b700 1 --2- 172.21.15.17:0/1143881717 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fe15057430 0x55fe15066ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.730 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.732+0000 7f51ad39b700 1 -- 172.21.15.17:0/1143881717 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51980098e0 con 0x55fe1504d480 2023-07-14T12:01:18.730 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.733+0000 7f51acb9a700 1 --2- 172.21.15.17:0/1143881717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fe1504ddd0 0x55fe1505fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T12:01:18.730 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.733+0000 7f51ad39b700 1 --2- 172.21.15.17:0/1143881717 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe1504d480 0x55fe1505f920 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f51a4096c60 tx=0x7f51a4003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3518fe05a336f7da server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:18.731 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.733+0000 7f5190ff9700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-07-14T12:01:18.731 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.733+0000 7f5190ff9700 1 -- 172.21.15.17:0/1143881717 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a409e220 con 0x55fe1504d480 2023-07-14T12:01:18.731 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.733+0000 7f5190ff9700 1 -- 172.21.15.17:0/1143881717 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f51a40a2640 con 0x55fe1504d480 2023-07-14T12:01:18.731 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.733+0000 7f51c2edbb80 1 -- 172.21.15.17:0/1143881717 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fe14ef84a0 con 0x55fe1504d480 2023-07-14T12:01:18.731 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.733+0000 7f5190ff9700 1 -- 172.21.15.17:0/1143881717 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51a40983a0 con 0x55fe1504d480 2023-07-14T12:01:18.731 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.733+0000 7f51c2edbb80 10 client.15084 fetch_fsmap learned FSMap version 27 2023-07-14T12:01:18.731 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.733+0000 7f51c2edbb80 1 -- 172.21.15.17:0/1143881717 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f51a819dee0 con 0x55fe1504d480 2023-07-14T12:01:18.732 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.733+0000 7f5190ff9700 1 -- 172.21.15.17:0/1143881717 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fe14ef84a0 con 0x55fe1504d480 2023-07-14T12:01:18.732 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.734+0000 7f5190ff9700 1 -- 172.21.15.17:0/1143881717 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f51a40a21b0 con 0x55fe1504d480 2023-07-14T12:01:18.732 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.734+0000 7f51c2edbb80 10 client.15084 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T12:01:18.732 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.734+0000 7f51c2edbb80 10 client.15084 resolve_mds: resolved name 'c' to daemon mds.c 2023-07-14T12:01:18.732 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.734+0000 7f51c2edbb80 20 client.15084 populate_metadata read hostname 'smithi017' 2023-07-14T12:01:18.732 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.734+0000 7f51c2edbb80 1 --2- 172.21.15.17:0/1143881717 >> [v2:172.21.15.97:6836/3684187131,v1:172.21.15.97:6837/3684187131] conn(0x55fe1506adb0 0x55fe15069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:18.732 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.734+0000 7f51c2edbb80 4 client.15084 mds_command: new command op to 24517 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T12:01:18.732 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.734+0000 7f51c2edbb80 1 -- 172.21.15.17:0/1143881717 --> [v2:172.21.15.97:6836/3684187131,v1:172.21.15.97:6837/3684187131] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe14f97e60 con 0x55fe1506adb0 2023-07-14T12:01:18.733 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.734+0000 7f51acb9a700 1 --2- 172.21.15.17:0/1143881717 >> [v2:172.21.15.97:6836/3684187131,v1:172.21.15.97:6837/3684187131] conn(0x55fe1506adb0 0x55fe15069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:18.733 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.735+0000 7f51acb9a700 1 --2- 172.21.15.17:0/1143881717 >> [v2:172.21.15.97:6836/3684187131,v1:172.21.15.97:6837/3684187131] conn(0x55fe1506adb0 0x55fe15069dd0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f5add6a8f91c55fe server_cookie=ce72d0c5ee090f7d in_seq=0 out_seq=0 2023-07-14T12:01:18.733 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.735+0000 7f5190ff9700 10 client.15084 ms_handle_connect on v2:172.21.15.97:6836/3684187131 2023-07-14T12:01:18.733 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.735+0000 7f5190ff9700 1 -- 172.21.15.17:0/1143881717 <== mds.4 v2:172.21.15.97:6836/3684187131 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55fe14f97e60 con 0x55fe1506adb0 2023-07-14T12:01:18.733 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.735+0000 7f5190ff9700 10 client.15084 handle_command_reply: tid=0 2023-07-14T12:01:18.733 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.735+0000 7f51c2edbb80 1 client.15084 shutdown 2023-07-14T12:01:18.733 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.736+0000 7f51c2edbb80 1 -- 172.21.15.17:0/1143881717 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe1504d480 msgr2=0x55fe1505f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:18.733 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.736+0000 7f51c2edbb80 1 --2- 172.21.15.17:0/1143881717 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe1504d480 0x55fe1505f920 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f51a4096c60 tx=0x7f51a4003f70 comp rx=0 tx=0).stop 2023-07-14T12:01:18.734 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.736+0000 7f51c2edbb80 1 -- 172.21.15.17:0/1143881717 shutdown_connections 2023-07-14T12:01:18.734 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.736+0000 7f51c2edbb80 1 --2- 172.21.15.17:0/1143881717 >> [v2:172.21.15.97:6836/3684187131,v1:172.21.15.97:6837/3684187131] conn(0x55fe1506adb0 0x55fe15069dd0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.734 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.736+0000 7f51c2edbb80 1 --2- 172.21.15.17:0/1143881717 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fe15057430 0x55fe15066ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.734 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.736+0000 7f5190ff9700 0 client.15084 ms_handle_reset on v2:172.21.15.97:6836/3684187131 2023-07-14T12:01:18.734 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.736+0000 7f51c2edbb80 1 --2- 172.21.15.17:0/1143881717 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe1504d480 0x55fe1505f920 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.734 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.736+0000 7f51c2edbb80 1 --2- 172.21.15.17:0/1143881717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fe1504ddd0 0x55fe1505fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.734 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.736+0000 7f51c2edbb80 1 -- 172.21.15.17:0/1143881717 >> 172.21.15.17:0/1143881717 conn(0x55fe15042d50 msgr2=0x55fe15047860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:18.735 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.736+0000 7f51c2edbb80 1 -- 172.21.15.17:0/1143881717 shutdown_connections 2023-07-14T12:01:18.735 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.736+0000 7f51c2edbb80 1 -- 172.21.15.17:0/1143881717 wait complete. 2023-07-14T12:01:18.735 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.736+0000 7f51c2edbb80 20 client.15084 trim_cache size 0 max 16384 2023-07-14T12:01:18.736 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.755+0000 7f51c2edbb80 1 Processor -- start 2023-07-14T12:01:18.736 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.755+0000 7f51c2edbb80 1 -- start start 2023-07-14T12:01:18.736 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.755+0000 7f51c2edbb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fe1504d480 0x55fe150734f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:18.736 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.755+0000 7f51c2edbb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fe1504ddd0 0x55fe15066e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:18.737 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.755+0000 7f51c2edbb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe15057430 0x55fe1505e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:18.737 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.756+0000 7f51c2edbb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe14f484b0 con 0x55fe15057430 2023-07-14T12:01:18.737 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.756+0000 7f51c2edbb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe14f6d7c0 con 0x55fe1504ddd0 2023-07-14T12:01:18.737 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.756+0000 7f51c2edbb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe14f5d480 con 0x55fe1504d480 2023-07-14T12:01:18.737 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.756+0000 7f51ad39b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fe1504d480 0x55fe150734f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:18.737 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.756+0000 7f51ad39b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fe1504d480 0x55fe150734f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.17:54212/0 (socket says 172.21.15.17:54212) 2023-07-14T12:01:18.737 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.756+0000 7f51ad39b700 1 -- 172.21.15.17:0/1068316490 learned_addr learned my addr 172.21.15.17:0/1068316490 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:18.737 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.756+0000 7f51acb9a700 1 --2- 172.21.15.17:0/1068316490 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fe1504ddd0 0x55fe15066e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:18.738 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.756+0000 7f51adb9c700 1 --2- 172.21.15.17:0/1068316490 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe15057430 0x55fe1505e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:18.738 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.756+0000 7f51ad39b700 1 -- 172.21.15.17:0/1068316490 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fe1504ddd0 msgr2=0x55fe15066e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T12:01:18.738 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.756+0000 7f51ad39b700 1 --2- 172.21.15.17:0/1068316490 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fe1504ddd0 0x55fe15066e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.738 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.756+0000 7f51ad39b700 1 -- 172.21.15.17:0/1068316490 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe15057430 msgr2=0x55fe1505e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T12:01:18.738 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.756+0000 7f51ad39b700 1 --2- 172.21.15.17:0/1068316490 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe15057430 0x55fe1505e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.738 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.756+0000 7f51ad39b700 1 -- 172.21.15.17:0/1068316490 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe15060af0 con 0x55fe1504d480 2023-07-14T12:01:18.740 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.756+0000 7f51ad39b700 1 --2- 172.21.15.17:0/1068316490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fe1504d480 0x55fe150734f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f51a40a2110 tx=0x7f51a4003680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=399f4d912cbe9ef7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:18.740 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.757+0000 7f5193fff700 1 -- 172.21.15.17:0/1068316490 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a4004e80 con 0x55fe1504d480 2023-07-14T12:01:18.741 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.757+0000 7f5193fff700 1 -- 172.21.15.17:0/1068316490 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51a400ce40 con 0x55fe1504d480 2023-07-14T12:01:18.741 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.757+0000 7f5193fff700 1 -- 172.21.15.17:0/1068316490 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a4098330 con 0x55fe1504d480 2023-07-14T12:01:18.741 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.757+0000 7f51c2edbb80 1 -- 172.21.15.17:0/1068316490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fe1504d480 msgr2=0x55fe150734f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:18.741 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.757+0000 7f51c2edbb80 1 --2- 172.21.15.17:0/1068316490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fe1504d480 0x55fe150734f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f51a40a2110 tx=0x7f51a4003680 comp rx=0 tx=0).stop 2023-07-14T12:01:18.741 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.757+0000 7f51c2edbb80 1 -- 172.21.15.17:0/1068316490 shutdown_connections 2023-07-14T12:01:18.741 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.757+0000 7f51c2edbb80 1 --2- 172.21.15.17:0/1068316490 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe15057430 0x55fe1505e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.741 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.757+0000 7f51c2edbb80 1 --2- 172.21.15.17:0/1068316490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fe1504d480 0x55fe150734f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.742 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.757+0000 7f51c2edbb80 1 --2- 172.21.15.17:0/1068316490 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fe1504ddd0 0x55fe15066e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.742 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.757+0000 7f51c2edbb80 1 -- 172.21.15.17:0/1068316490 >> 172.21.15.17:0/1068316490 conn(0x55fe15042d50 msgr2=0x55fe150442c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:18.742 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.757+0000 7f51c2edbb80 1 -- 172.21.15.17:0/1068316490 shutdown_connections 2023-07-14T12:01:18.742 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.757+0000 7f51c2edbb80 1 -- 172.21.15.17:0/1068316490 wait complete. 2023-07-14T12:01:18.742 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.758+0000 7f51c2edbb80 1 -- start start 2023-07-14T12:01:18.742 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.758+0000 7f51c2edbb80 1 Processor -- start 2023-07-14T12:01:18.742 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.759+0000 7f51c2edbb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fe1504d480 0x55fe1505c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:18.742 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.759+0000 7f51c2edbb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe1504ddd0 0x55fe15066dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:18.743 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.759+0000 7f51c2edbb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fe15057430 0x55fe1505ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:18.743 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.759+0000 7f51c2edbb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe1505d020 con 0x55fe1504ddd0 2023-07-14T12:01:18.743 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.759+0000 7f51c2edbb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe150be620 con 0x55fe15057430 2023-07-14T12:01:18.743 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.759+0000 7f51c2edbb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe150be760 con 0x55fe1504d480 2023-07-14T12:01:18.743 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.759+0000 7f51c2edbb80 10 client.0 _ll_register_callbacks cb 0x7f51a0023030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T12:01:18.743 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.759+0000 7f51acb9a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe1504ddd0 0x55fe15066dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:18.743 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.759+0000 7f51ad39b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fe1504d480 0x55fe1505c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:18.744 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.759+0000 7f51acb9a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe1504ddd0 0x55fe15066dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53628/0 (socket says 172.21.15.17:53628) 2023-07-14T12:01:18.744 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.759+0000 7f51ad39b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fe1504d480 0x55fe1505c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.17:54222/0 (socket says 172.21.15.17:54222) 2023-07-14T12:01:18.744 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.759+0000 7f51acb9a700 1 -- 172.21.15.17:0/3934767929 learned_addr learned my addr 172.21.15.17:0/3934767929 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:18.744 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.759+0000 7f51adb9c700 1 --2- 172.21.15.17:0/3934767929 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fe15057430 0x55fe1505ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:18.744 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.760+0000 7f51acb9a700 1 -- 172.21.15.17:0/3934767929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fe1504d480 msgr2=0x55fe1505c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T12:01:18.744 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.760+0000 7f51acb9a700 1 --2- 172.21.15.17:0/3934767929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fe1504d480 0x55fe1505c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.744 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.760+0000 7f51acb9a700 1 -- 172.21.15.17:0/3934767929 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fe15057430 msgr2=0x55fe1505ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T12:01:18.744 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.760+0000 7f51acb9a700 1 --2- 172.21.15.17:0/3934767929 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fe15057430 0x55fe1505ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.745 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.760+0000 7f51acb9a700 1 -- 172.21.15.17:0/3934767929 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51a819dee0 con 0x55fe1504ddd0 2023-07-14T12:01:18.745 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.760+0000 7f51acb9a700 1 --2- 172.21.15.17:0/3934767929 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe1504ddd0 0x55fe15066dc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x55fe1506bb10 tx=0x7f5198021e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce65fe87dabe8f84 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:18.745 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.760+0000 7f5190ff9700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-07-14T12:01:18.745 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.760+0000 7f5190ff9700 1 -- 172.21.15.17:0/3934767929 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5198017040 con 0x55fe1504ddd0 2023-07-14T12:01:18.745 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.760+0000 7f5190ff9700 1 -- 172.21.15.17:0/3934767929 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f5198012db0 con 0x55fe1504ddd0 2023-07-14T12:01:18.745 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.760+0000 7f51c2edbb80 1 -- 172.21.15.17:0/3934767929 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fe14f5a000 con 0x55fe1504ddd0 2023-07-14T12:01:18.745 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.760+0000 7f5190ff9700 1 -- 172.21.15.17:0/3934767929 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5198004030 con 0x55fe1504ddd0 2023-07-14T12:01:18.745 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.761+0000 7f51c2edbb80 10 client.15090 fetch_fsmap learned FSMap version 27 2023-07-14T12:01:18.746 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.761+0000 7f51c2edbb80 1 -- 172.21.15.17:0/3934767929 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55fe14db5c10 con 0x55fe1504ddd0 2023-07-14T12:01:18.747 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.761+0000 7f5190ff9700 1 -- 172.21.15.17:0/3934767929 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fe14f5a000 con 0x55fe1504ddd0 2023-07-14T12:01:18.747 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.761+0000 7f5190ff9700 1 -- 172.21.15.17:0/3934767929 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f51980037a0 con 0x55fe1504ddd0 2023-07-14T12:01:18.747 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.761+0000 7f51c2edbb80 10 client.15090 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T12:01:18.747 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.761+0000 7f51c2edbb80 10 client.15090 resolve_mds: resolved name 'c' to daemon mds.c 2023-07-14T12:01:18.747 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.761+0000 7f51c2edbb80 20 client.15090 populate_metadata read hostname 'smithi017' 2023-07-14T12:01:18.748 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.761+0000 7f51c2edbb80 1 --2- 172.21.15.17:0/3934767929 >> [v2:172.21.15.97:6836/3684187131,v1:172.21.15.97:6837/3684187131] conn(0x55fe150c34d0 0x55fe150bfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:18.748 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.761+0000 7f51c2edbb80 4 client.15090 mds_command: new command op to 24517 tid=0[{"prefix": "status"}] 2023-07-14T12:01:18.748 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.761+0000 7f51c2edbb80 1 -- 172.21.15.17:0/3934767929 --> [v2:172.21.15.97:6836/3684187131,v1:172.21.15.97:6837/3684187131] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fe14dedf20 con 0x55fe150c34d0 2023-07-14T12:01:18.748 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.761+0000 7f51ad39b700 1 --2- 172.21.15.17:0/3934767929 >> [v2:172.21.15.97:6836/3684187131,v1:172.21.15.97:6837/3684187131] conn(0x55fe150c34d0 0x55fe150bfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:18.748 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.762+0000 7f51ad39b700 1 --2- 172.21.15.17:0/3934767929 >> [v2:172.21.15.97:6836/3684187131,v1:172.21.15.97:6837/3684187131] conn(0x55fe150c34d0 0x55fe150bfe00 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=10904a609fc53409 server_cookie=e0b6bb9476d2ad6a in_seq=0 out_seq=0 2023-07-14T12:01:18.748 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.762+0000 7f5190ff9700 10 client.15090 ms_handle_connect on v2:172.21.15.97:6836/3684187131 2023-07-14T12:01:18.748 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.762+0000 7f5190ff9700 1 -- 172.21.15.17:0/3934767929 <== mds.4 v2:172.21.15.97:6836/3684187131 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55fe14dedf20 con 0x55fe150c34d0 2023-07-14T12:01:18.749 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.762+0000 7f5190ff9700 10 client.15090 handle_command_reply: tid=0 2023-07-14T12:01:18.749 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.762+0000 7f51c2edbb80 1 client.15090 shutdown 2023-07-14T12:01:18.749 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.762+0000 7f51c2edbb80 1 -- 172.21.15.17:0/3934767929 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe1504ddd0 msgr2=0x55fe15066dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:18.749 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.762+0000 7f51c2edbb80 1 --2- 172.21.15.17:0/3934767929 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe1504ddd0 0x55fe15066dc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x55fe1506bb10 tx=0x7f5198021e90 comp rx=0 tx=0).stop 2023-07-14T12:01:18.749 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.762+0000 7f51c2edbb80 1 -- 172.21.15.17:0/3934767929 shutdown_connections 2023-07-14T12:01:18.749 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.762+0000 7f51c2edbb80 1 --2- 172.21.15.17:0/3934767929 >> [v2:172.21.15.97:6836/3684187131,v1:172.21.15.97:6837/3684187131] conn(0x55fe150c34d0 0x55fe150bfe00 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.749 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.762+0000 7f51c2edbb80 1 --2- 172.21.15.17:0/3934767929 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fe15057430 0x55fe1505ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.750 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.762+0000 7f51c2edbb80 1 --2- 172.21.15.17:0/3934767929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fe1504d480 0x55fe1505c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.750 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.762+0000 7f51c2edbb80 1 --2- 172.21.15.17:0/3934767929 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fe1504ddd0 0x55fe15066dc0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.750 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.762+0000 7f5190ff9700 0 client.15090 ms_handle_reset on v2:172.21.15.97:6836/3684187131 2023-07-14T12:01:18.750 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.762+0000 7f51c2edbb80 1 -- 172.21.15.17:0/3934767929 >> 172.21.15.17:0/3934767929 conn(0x55fe15042d50 msgr2=0x55fe150734f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:18.750 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.763+0000 7f51c2edbb80 1 -- 172.21.15.17:0/3934767929 shutdown_connections 2023-07-14T12:01:18.750 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.763+0000 7f51c2edbb80 1 -- 172.21.15.17:0/3934767929 wait complete. 2023-07-14T12:01:18.750 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.763+0000 7f51c2edbb80 20 client.15090 trim_cache size 0 max 16384 2023-07-14T12:01:18.751 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2023-07-14T12:01:18.751 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "b3c386cc-223c-11ee-9b33-001a4aab830c", 2023-07-14T12:01:18.751 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 4, 2023-07-14T12:01:18.751 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 24517, 2023-07-14T12:01:18.751 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2023-07-14T12:01:18.751 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2023-07-14T12:01:18.752 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2023-07-14T12:01:18.752 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 62.277663304000001, 2023-07-14T12:01:18.752 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 27, 2023-07-14T12:01:18.752 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 92, 2023-07-14T12:01:18.752 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 90, 2023-07-14T12:01:18.752 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 114.29737470400001 2023-07-14T12:01:18.752 INFO:teuthology.orchestra.run.smithi017.stdout:} 2023-07-14T12:01:18.753 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.763+0000 7f51af5ff700 1 -- 172.21.15.17:0/2270205062 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f51940a11d0 msgr2=0x7f51940a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:18.753 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.763+0000 7f51af5ff700 1 --2- 172.21.15.17:0/2270205062 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f51940a11d0 0x7f51940a3680 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f51a000b780 tx=0x7f51a00095a0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.753 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.763+0000 7f51af5ff700 1 -- 172.21.15.17:0/2270205062 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51a80ffbc0 msgr2=0x7f51a81959e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:18.753 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.763+0000 7f51af5ff700 1 --2- 172.21.15.17:0/2270205062 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51a80ffbc0 0x7f51a81959e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f51a400eb30 tx=0x7f51a400ee40 comp rx=0 tx=0).stop 2023-07-14T12:01:18.754 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.764+0000 7f51af5ff700 1 -- 172.21.15.17:0/2270205062 shutdown_connections 2023-07-14T12:01:18.754 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.764+0000 7f51af5ff700 1 --2- 172.21.15.17:0/2270205062 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f51940a11d0 0x7f51940a3680 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.754 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.764+0000 7f51af5ff700 1 --2- 172.21.15.17:0/2270205062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51a8101800 0x7f51a819cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.754 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.764+0000 7f51af5ff700 1 --2- 172.21.15.17:0/2270205062 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51a80ffbc0 0x7f51a81959e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.754 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.764+0000 7f51af5ff700 1 --2- 172.21.15.17:0/2270205062 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51a8100e50 0x7f51a8195f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.754 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.764+0000 7f51af5ff700 1 -- 172.21.15.17:0/2270205062 >> 172.21.15.17:0/2270205062 conn(0x7f51a80fb200 msgr2=0x7f51a81042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:18.754 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.764+0000 7f51af5ff700 1 -- 172.21.15.17:0/2270205062 shutdown_connections 2023-07-14T12:01:18.755 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.764+0000 7f51af5ff700 1 -- 172.21.15.17:0/2270205062 wait complete. 2023-07-14T12:01:18.762 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.g status 2023-07-14T12:01:18.969 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.988+0000 7fadeadff700 1 -- 172.21.15.17:0/2069145465 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fade4101760 msgr2=0x7fade4108020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:18.969 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.988+0000 7fadeadff700 1 --2- 172.21.15.17:0/2069145465 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fade4101760 0x7fade4108020 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7faddc00b0a0 tx=0x7faddc00b3b0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.970 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.988+0000 7fadeadff700 1 -- 172.21.15.17:0/2069145465 shutdown_connections 2023-07-14T12:01:18.970 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.988+0000 7fadeadff700 1 --2- 172.21.15.17:0/2069145465 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fade40ffbe0 0x7fade40ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.970 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.988+0000 7fadeadff700 1 --2- 172.21.15.17:0/2069145465 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fade4101760 0x7fade4108020 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.970 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.988+0000 7fadeadff700 1 --2- 172.21.15.17:0/2069145465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fade4100dd0 0x7fade4101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.970 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.988+0000 7fadeadff700 1 -- 172.21.15.17:0/2069145465 >> 172.21.15.17:0/2069145465 conn(0x7fade40fb200 msgr2=0x7fade40fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:18.970 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.988+0000 7fadeadff700 1 -- 172.21.15.17:0/2069145465 shutdown_connections 2023-07-14T12:01:18.971 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.988+0000 7fadeadff700 1 -- 172.21.15.17:0/2069145465 wait complete. 2023-07-14T12:01:18.972 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.989+0000 7fadeadff700 1 Processor -- start 2023-07-14T12:01:18.972 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.989+0000 7fadeadff700 1 -- start start 2023-07-14T12:01:18.972 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.990+0000 7fadeadff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fade40ffbe0 0x7fade40742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:18.973 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.990+0000 7fadeadff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fade4100dd0 0x7fade4072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:18.973 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.990+0000 7fadeadff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fade4101760 0x7fade4072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:18.973 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.990+0000 7fadeadff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade40735e0 con 0x7fade40ffbe0 2023-07-14T12:01:18.973 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.990+0000 7fadeadff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade4073750 con 0x7fade4100dd0 2023-07-14T12:01:18.973 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.990+0000 7fadeadff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade419fa80 con 0x7fade4101760 2023-07-14T12:01:18.973 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.990+0000 7fade8b9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fade40ffbe0 0x7fade40742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:18.973 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.990+0000 7fade939c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fade4101760 0x7fade4072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:18.974 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.990+0000 7fade8b9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fade40ffbe0 0x7fade40742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53638/0 (socket says 172.21.15.17:53638) 2023-07-14T12:01:18.974 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.990+0000 7fade8b9b700 1 -- 172.21.15.17:0/367740119 learned_addr learned my addr 172.21.15.17:0/367740119 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:18.974 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.990+0000 7fade3fff700 1 --2- 172.21.15.17:0/367740119 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fade4100dd0 0x7fade4072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:18.974 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.990+0000 7fade939c700 1 -- 172.21.15.17:0/367740119 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fade4100dd0 msgr2=0x7fade4072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:18.974 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.990+0000 7fade939c700 1 --2- 172.21.15.17:0/367740119 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fade4100dd0 0x7fade4072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.974 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.990+0000 7fade939c700 1 -- 172.21.15.17:0/367740119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fade40ffbe0 msgr2=0x7fade40742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:18.974 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.990+0000 7fade939c700 1 --2- 172.21.15.17:0/367740119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fade40ffbe0 0x7fade40742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.975 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.990+0000 7fade939c700 1 -- 172.21.15.17:0/367740119 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faddc009d00 con 0x7fade4101760 2023-07-14T12:01:18.975 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.990+0000 7fade8b9b700 1 --2- 172.21.15.17:0/367740119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fade40ffbe0 0x7fade40742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T12:01:18.975 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.991+0000 7fade939c700 1 --2- 172.21.15.17:0/367740119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fade4101760 0x7fade4072e40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7faddc00b070 tx=0x7faddc00bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:18.975 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.991+0000 7fade1ffb700 1 -- 172.21.15.17:0/367740119 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc00e040 con 0x7fade4101760 2023-07-14T12:01:18.975 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.991+0000 7fadeadff700 1 -- 172.21.15.17:0/367740119 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fade419fd00 con 0x7fade4101760 2023-07-14T12:01:18.975 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.991+0000 7fade1ffb700 1 -- 172.21.15.17:0/367740119 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faddc009480 con 0x7fade4101760 2023-07-14T12:01:18.975 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.991+0000 7fade1ffb700 1 -- 172.21.15.17:0/367740119 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc0052a0 con 0x7fade4101760 2023-07-14T12:01:18.975 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.991+0000 7fadeadff700 1 -- 172.21.15.17:0/367740119 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fade41a0180 con 0x7fade4101760 2023-07-14T12:01:18.977 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.994+0000 7fade1ffb700 1 -- 172.21.15.17:0/367740119 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7faddc01d480 con 0x7fade4101760 2023-07-14T12:01:18.977 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.995+0000 7fadfe6fab80 1 Processor -- start 2023-07-14T12:01:18.977 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.995+0000 7fadfe6fab80 1 -- start start 2023-07-14T12:01:18.978 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.996+0000 7fade1ffb700 1 --2- 172.21.15.17:0/367740119 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fadd40a1150 0x7fadd40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:18.978 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.996+0000 7fadfe6fab80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555a38868480 0x555a38868890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:18.978 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.996+0000 7fadfe6fab80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a38868dd0 0x555a38869690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:18.978 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.996+0000 7fadfe6fab80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a38869c20 0x555a3886a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:18.978 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.996+0000 7fadfe6fab80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a3876dc60 con 0x555a38868480 2023-07-14T12:01:18.978 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.996+0000 7fadfe6fab80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a387c09f0 con 0x555a38868dd0 2023-07-14T12:01:18.978 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.996+0000 7fadfe6fab80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a387b9e80 con 0x555a38869c20 2023-07-14T12:01:18.979 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.996+0000 7fade1ffb700 1 -- 172.21.15.17:0/367740119 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7faddc0cdc50 con 0x7fade4101760 2023-07-14T12:01:18.979 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.996+0000 7fade3fff700 1 --2- 172.21.15.17:0/367740119 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fadd40a1150 0x7fadd40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:18.979 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.996+0000 7fade8b9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555a38868480 0x555a38868890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:18.979 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.996+0000 7fade8b9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555a38868480 0x555a38868890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53648/0 (socket says 172.21.15.17:53648) 2023-07-14T12:01:18.979 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.996+0000 7fade8b9b700 1 -- 172.21.15.17:0/1929721454 learned_addr learned my addr 172.21.15.17:0/1929721454 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:18.979 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.996+0000 7fade3fff700 1 --2- 172.21.15.17:0/1929721454 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a38868dd0 0x555a38869690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:18.979 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.996+0000 7fade939c700 1 --2- 172.21.15.17:0/1929721454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a38869c20 0x555a3886a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:18.979 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.997+0000 7fade8b9b700 1 -- 172.21.15.17:0/1929721454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a38869c20 msgr2=0x555a3886a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T12:01:18.980 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.997+0000 7fade8b9b700 1 --2- 172.21.15.17:0/1929721454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a38869c20 0x555a3886a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.980 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.997+0000 7fade8b9b700 1 -- 172.21.15.17:0/1929721454 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a38868dd0 msgr2=0x555a38869690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T12:01:18.980 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.997+0000 7fade8b9b700 1 --2- 172.21.15.17:0/1929721454 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a38868dd0 0x555a38869690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.980 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.997+0000 7fade8b9b700 1 -- 172.21.15.17:0/1929721454 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a385e7d90 con 0x555a38868480 2023-07-14T12:01:18.980 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.997+0000 7fade3fff700 1 --2- 172.21.15.17:0/367740119 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fadd40a1150 0x7fadd40a3600 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fadd00139d0 tx=0x7fadd0009b60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:18.980 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.997+0000 7fade8b9b700 1 --2- 172.21.15.17:0/1929721454 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555a38868480 0x555a38868890 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fadd8011650 tx=0x7fadd8011960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c184b7325a1ee171 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:18.980 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.997+0000 7fadceffd700 1 -- 172.21.15.17:0/1929721454 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd8004a30 con 0x555a38868480 2023-07-14T12:01:18.981 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.997+0000 7fadceffd700 1 -- 172.21.15.17:0/1929721454 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadd80054e0 con 0x555a38868480 2023-07-14T12:01:18.981 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.997+0000 7fadceffd700 1 -- 172.21.15.17:0/1929721454 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd8019720 con 0x555a38868480 2023-07-14T12:01:18.984 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.998+0000 7fadfe6fab80 1 -- 172.21.15.17:0/1929721454 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555a38868480 msgr2=0x555a38868890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:18.984 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.998+0000 7fadfe6fab80 1 --2- 172.21.15.17:0/1929721454 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555a38868480 0x555a38868890 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fadd8011650 tx=0x7fadd8011960 comp rx=0 tx=0).stop 2023-07-14T12:01:18.984 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.998+0000 7fadfe6fab80 1 -- 172.21.15.17:0/1929721454 shutdown_connections 2023-07-14T12:01:18.984 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.998+0000 7fadfe6fab80 1 --2- 172.21.15.17:0/1929721454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a38869c20 0x555a3886a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.984 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.998+0000 7fadfe6fab80 1 --2- 172.21.15.17:0/1929721454 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555a38868480 0x555a38868890 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.984 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.998+0000 7fadfe6fab80 1 --2- 172.21.15.17:0/1929721454 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a38868dd0 0x555a38869690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.985 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.998+0000 7fadfe6fab80 1 -- 172.21.15.17:0/1929721454 >> 172.21.15.17:0/1929721454 conn(0x555a3885dd50 msgr2=0x555a38863180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:18.985 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.998+0000 7fadfe6fab80 1 -- 172.21.15.17:0/1929721454 shutdown_connections 2023-07-14T12:01:18.985 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.998+0000 7fadfe6fab80 1 -- 172.21.15.17:0/1929721454 wait complete. 2023-07-14T12:01:18.985 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.999+0000 7fadfe6fab80 1 -- start start 2023-07-14T12:01:18.985 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.999+0000 7fadfe6fab80 1 Processor -- start 2023-07-14T12:01:18.985 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.999+0000 7fadfe6fab80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a38868480 0x555a38872070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:18.985 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.999+0000 7fadfe6fab80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a38868dd0 0x555a388725b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:18.986 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.999+0000 7fadfe6fab80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555a38869c20 0x555a388795f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:18.986 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.999+0000 7fadfe6fab80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a38759a70 con 0x555a38869c20 2023-07-14T12:01:18.986 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.999+0000 7fadfe6fab80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a38759770 con 0x555a38868480 2023-07-14T12:01:18.986 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.999+0000 7fadfe6fab80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a3863d8d0 con 0x555a38868dd0 2023-07-14T12:01:18.986 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:18.999+0000 7fadfe6fab80 10 client.0 _ll_register_callbacks cb 0x555a38757710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T12:01:18.986 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.000+0000 7fade939c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555a38869c20 0x555a388795f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:18.986 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.000+0000 7fade8b9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a38868480 0x555a38872070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:18.986 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.000+0000 7fade939c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555a38869c20 0x555a388795f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53662/0 (socket says 172.21.15.17:53662) 2023-07-14T12:01:18.987 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.000+0000 7fade8b9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a38868480 0x555a38872070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.17:54194/0 (socket says 172.21.15.17:54194) 2023-07-14T12:01:18.987 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.000+0000 7fade939c700 1 -- 172.21.15.17:0/3714120275 learned_addr learned my addr 172.21.15.17:0/3714120275 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:18.987 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.000+0000 7fade3fff700 1 --2- 172.21.15.17:0/3714120275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a38868dd0 0x555a388725b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:18.987 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.000+0000 7fade939c700 1 -- 172.21.15.17:0/3714120275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a38868dd0 msgr2=0x555a388725b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T12:01:18.987 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.000+0000 7fade939c700 1 --2- 172.21.15.17:0/3714120275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a38868dd0 0x555a388725b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.987 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.000+0000 7fade939c700 1 -- 172.21.15.17:0/3714120275 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a38868480 msgr2=0x555a38872070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T12:01:18.987 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.000+0000 7fade939c700 1 --2- 172.21.15.17:0/3714120275 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a38868480 0x555a38872070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.987 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.000+0000 7fade939c700 1 -- 172.21.15.17:0/3714120275 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadd8009e10 con 0x555a38869c20 2023-07-14T12:01:18.988 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.000+0000 7fade8b9b700 1 --2- 172.21.15.17:0/3714120275 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a38868480 0x555a38872070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T12:01:18.988 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.000+0000 7fade939c700 1 --2- 172.21.15.17:0/3714120275 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555a38869c20 0x555a388795f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7faddc00d010 tx=0x7faddc0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f4d2a772909df37 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:18.988 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.000+0000 7fadc3fff700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-07-14T12:01:18.988 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.000+0000 7fadc3fff700 1 -- 172.21.15.17:0/3714120275 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc09b1a0 con 0x555a38869c20 2023-07-14T12:01:18.988 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.001+0000 7fadc3fff700 1 -- 172.21.15.17:0/3714120275 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7faddc09ad40 con 0x555a38869c20 2023-07-14T12:01:18.988 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.001+0000 7fadfe6fab80 1 -- 172.21.15.17:0/3714120275 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555a387134a0 con 0x555a38869c20 2023-07-14T12:01:18.988 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.001+0000 7fadc3fff700 1 -- 172.21.15.17:0/3714120275 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7faddc09ed70 con 0x555a38869c20 2023-07-14T12:01:18.989 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.001+0000 7fadfe6fab80 10 client.15102 fetch_fsmap learned FSMap version 27 2023-07-14T12:01:18.989 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.001+0000 7fadfe6fab80 1 -- 172.21.15.17:0/3714120275 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555a385d0c10 con 0x555a38869c20 2023-07-14T12:01:18.989 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.001+0000 7fadc3fff700 1 -- 172.21.15.17:0/3714120275 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555a387134a0 con 0x555a38869c20 2023-07-14T12:01:18.989 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.001+0000 7fadc3fff700 1 -- 172.21.15.17:0/3714120275 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7faddc09a230 con 0x555a38869c20 2023-07-14T12:01:18.989 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.001+0000 7fadfe6fab80 10 client.15102 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T12:01:18.989 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.001+0000 7fadfe6fab80 10 client.15102 resolve_mds: resolved name 'g' to daemon mds.g 2023-07-14T12:01:18.989 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.001+0000 7fadfe6fab80 20 client.15102 populate_metadata read hostname 'smithi017' 2023-07-14T12:01:18.989 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.001+0000 7fadfe6fab80 1 --2- 172.21.15.17:0/3714120275 >> [v2:172.21.15.17:6838/2976493015,v1:172.21.15.17:6839/2976493015] conn(0x555a3887ce80 0x555a3887c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:18.990 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.001+0000 7fadfe6fab80 4 client.15102 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T12:01:18.990 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.001+0000 7fadfe6fab80 1 -- 172.21.15.17:0/3714120275 --> [v2:172.21.15.17:6838/2976493015,v1:172.21.15.17:6839/2976493015] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555a387b2e60 con 0x555a3887ce80 2023-07-14T12:01:18.990 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.002+0000 7fade8b9b700 1 --2- 172.21.15.17:0/3714120275 >> [v2:172.21.15.17:6838/2976493015,v1:172.21.15.17:6839/2976493015] conn(0x555a3887ce80 0x555a3887c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:18.990 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.002+0000 7fade8b9b700 1 --2- 172.21.15.17:0/3714120275 >> [v2:172.21.15.17:6838/2976493015,v1:172.21.15.17:6839/2976493015] conn(0x555a3887ce80 0x555a3887c520 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=db129369d4d1cf3 server_cookie=1881bc12f8be7148 in_seq=0 out_seq=0 2023-07-14T12:01:18.990 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.002+0000 7fadc3fff700 10 client.15102 ms_handle_connect on v2:172.21.15.17:6838/2976493015 2023-07-14T12:01:18.990 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.003+0000 7fadc3fff700 1 -- 172.21.15.17:0/3714120275 <== mds.3 v2:172.21.15.17:6838/2976493015 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555a387b2e60 con 0x555a3887ce80 2023-07-14T12:01:18.990 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.003+0000 7fadc3fff700 10 client.15102 handle_command_reply: tid=0 2023-07-14T12:01:18.991 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.003+0000 7fadfe6fab80 1 client.15102 shutdown 2023-07-14T12:01:18.991 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.003+0000 7fadfe6fab80 1 -- 172.21.15.17:0/3714120275 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555a38869c20 msgr2=0x555a388795f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:18.991 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.003+0000 7fadfe6fab80 1 --2- 172.21.15.17:0/3714120275 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555a38869c20 0x555a388795f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7faddc00d010 tx=0x7faddc0099e0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.991 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.004+0000 7fadfe6fab80 1 -- 172.21.15.17:0/3714120275 shutdown_connections 2023-07-14T12:01:18.991 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.004+0000 7fadfe6fab80 1 --2- 172.21.15.17:0/3714120275 >> [v2:172.21.15.17:6838/2976493015,v1:172.21.15.17:6839/2976493015] conn(0x555a3887ce80 0x555a3887c520 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.991 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.004+0000 7fadfe6fab80 1 --2- 172.21.15.17:0/3714120275 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a38868480 0x555a38872070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.991 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.004+0000 7fadfe6fab80 1 --2- 172.21.15.17:0/3714120275 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555a38869c20 0x555a388795f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.991 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.004+0000 7fadfe6fab80 1 --2- 172.21.15.17:0/3714120275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a38868dd0 0x555a388725b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:18.992 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.004+0000 7fadfe6fab80 1 -- 172.21.15.17:0/3714120275 >> 172.21.15.17:0/3714120275 conn(0x555a3885dd50 msgr2=0x555a38862860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:18.992 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.004+0000 7fadc3fff700 0 client.15102 ms_handle_reset on v2:172.21.15.17:6838/2976493015 2023-07-14T12:01:18.992 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.004+0000 7fadfe6fab80 1 -- 172.21.15.17:0/3714120275 shutdown_connections 2023-07-14T12:01:18.992 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.004+0000 7fadfe6fab80 1 -- 172.21.15.17:0/3714120275 wait complete. 2023-07-14T12:01:18.992 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.004+0000 7fadfe6fab80 20 client.15102 trim_cache size 0 max 16384 2023-07-14T12:01:19.002 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.022+0000 7fadfe6fab80 1 Processor -- start 2023-07-14T12:01:19.003 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.023+0000 7fadfe6fab80 1 -- start start 2023-07-14T12:01:19.003 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.023+0000 7fadfe6fab80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555a38868480 0x555a38885c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.004 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.023+0000 7fadfe6fab80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a38868dd0 0x555a38879580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.004 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.023+0000 7fadfe6fab80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a38869c20 0x555a3887c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.004 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.023+0000 7fadfe6fab80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a38873260 con 0x555a38868480 2023-07-14T12:01:19.004 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.023+0000 7fadfe6fab80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a38763470 con 0x555a38868dd0 2023-07-14T12:01:19.004 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.023+0000 7fadfe6fab80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a38649de0 con 0x555a38869c20 2023-07-14T12:01:19.004 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.023+0000 7fade8b9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555a38868480 0x555a38885c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.004 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.023+0000 7fade939c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a38869c20 0x555a3887c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.005 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.023+0000 7fade3fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a38868dd0 0x555a38879580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.005 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.023+0000 7fade8b9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555a38868480 0x555a38885c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53674/0 (socket says 172.21.15.17:53674) 2023-07-14T12:01:19.005 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.023+0000 7fade939c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a38869c20 0x555a3887c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.17:54266/0 (socket says 172.21.15.17:54266) 2023-07-14T12:01:19.005 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.023+0000 7fade8b9b700 1 -- 172.21.15.17:0/1004671507 learned_addr learned my addr 172.21.15.17:0/1004671507 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:19.005 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.024+0000 7fade3fff700 1 -- 172.21.15.17:0/1004671507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a38869c20 msgr2=0x555a3887c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T12:01:19.005 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.024+0000 7fade3fff700 1 --2- 172.21.15.17:0/1004671507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a38869c20 0x555a3887c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.005 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.024+0000 7fade3fff700 1 -- 172.21.15.17:0/1004671507 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555a38868480 msgr2=0x555a38885c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T12:01:19.006 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.024+0000 7fade3fff700 1 --2- 172.21.15.17:0/1004671507 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555a38868480 0x555a38885c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.006 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.024+0000 7fade3fff700 1 -- 172.21.15.17:0/1004671507 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a3860e200 con 0x555a38868dd0 2023-07-14T12:01:19.006 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.024+0000 7fade939c700 1 --2- 172.21.15.17:0/1004671507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a38869c20 0x555a3887c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T12:01:19.006 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.024+0000 7fade8b9b700 1 --2- 172.21.15.17:0/1004671507 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555a38868480 0x555a38885c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T12:01:19.008 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.024+0000 7fade3fff700 1 --2- 172.21.15.17:0/1004671507 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a38868dd0 0x555a38879580 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fadd0009e20 tx=0x7fadd0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9028f48620f9a026 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:19.009 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.024+0000 7fadcf7fe700 1 -- 172.21.15.17:0/1004671507 <== mon.2 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd00044a0 con 0x555a38868dd0 2023-07-14T12:01:19.009 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.024+0000 7fadcf7fe700 1 -- 172.21.15.17:0/1004671507 <== mon.2 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadd0004cf0 con 0x555a38868dd0 2023-07-14T12:01:19.009 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.024+0000 7fadcf7fe700 1 -- 172.21.15.17:0/1004671507 <== mon.2 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd0005020 con 0x555a38868dd0 2023-07-14T12:01:19.009 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.025+0000 7fadfe6fab80 1 -- 172.21.15.17:0/1004671507 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a38868dd0 msgr2=0x555a38879580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:19.009 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.025+0000 7fadfe6fab80 1 --2- 172.21.15.17:0/1004671507 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a38868dd0 0x555a38879580 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fadd0009e20 tx=0x7fadd0009260 comp rx=0 tx=0).stop 2023-07-14T12:01:19.009 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.025+0000 7fadfe6fab80 1 -- 172.21.15.17:0/1004671507 shutdown_connections 2023-07-14T12:01:19.009 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.025+0000 7fadfe6fab80 1 --2- 172.21.15.17:0/1004671507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a38869c20 0x555a3887c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.010 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.025+0000 7fadfe6fab80 1 --2- 172.21.15.17:0/1004671507 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555a38868480 0x555a38885c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.010 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.025+0000 7fadfe6fab80 1 --2- 172.21.15.17:0/1004671507 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a38868dd0 0x555a38879580 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.010 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.025+0000 7fadfe6fab80 1 -- 172.21.15.17:0/1004671507 >> 172.21.15.17:0/1004671507 conn(0x555a3885dd50 msgr2=0x555a3885f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:19.010 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.025+0000 7fadfe6fab80 1 -- 172.21.15.17:0/1004671507 shutdown_connections 2023-07-14T12:01:19.010 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.025+0000 7fadfe6fab80 1 -- 172.21.15.17:0/1004671507 wait complete. 2023-07-14T12:01:19.010 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.025+0000 7fadfe6fab80 1 -- start start 2023-07-14T12:01:19.010 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.026+0000 7fadfe6fab80 1 Processor -- start 2023-07-14T12:01:19.011 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.026+0000 7fadfe6fab80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a38868480 0x555a38874780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.011 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.026+0000 7fadfe6fab80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555a38868dd0 0x555a38874cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.011 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.026+0000 7fadfe6fab80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a38869c20 0x555a388d0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.011 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.026+0000 7fadfe6fab80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a388d1280 con 0x555a38868dd0 2023-07-14T12:01:19.011 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.026+0000 7fadfe6fab80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a388d13c0 con 0x555a38868480 2023-07-14T12:01:19.011 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.026+0000 7fadfe6fab80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a388d1500 con 0x555a38869c20 2023-07-14T12:01:19.011 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.026+0000 7fadfe6fab80 10 client.0 _ll_register_callbacks cb 0x7faddc097170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T12:01:19.011 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.027+0000 7fade3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555a38868dd0 0x555a38874cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.012 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.027+0000 7fade8b9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a38868480 0x555a38874780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.012 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.027+0000 7fade3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555a38868dd0 0x555a38874cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53686/0 (socket says 172.21.15.17:53686) 2023-07-14T12:01:19.012 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.027+0000 7fade8b9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a38868480 0x555a38874780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.17:54220/0 (socket says 172.21.15.17:54220) 2023-07-14T12:01:19.012 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.027+0000 7fade3fff700 1 -- 172.21.15.17:0/934683754 learned_addr learned my addr 172.21.15.17:0/934683754 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:19.012 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.027+0000 7fade939c700 1 --2- 172.21.15.17:0/934683754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a38869c20 0x555a388d0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.012 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.027+0000 7fade3fff700 1 -- 172.21.15.17:0/934683754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a38869c20 msgr2=0x555a388d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T12:01:19.012 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.027+0000 7fade3fff700 1 --2- 172.21.15.17:0/934683754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a38869c20 0x555a388d0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.013 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.027+0000 7fade3fff700 1 -- 172.21.15.17:0/934683754 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a38868480 msgr2=0x555a38874780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T12:01:19.013 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.027+0000 7fade3fff700 1 --2- 172.21.15.17:0/934683754 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a38868480 0x555a38874780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.013 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.027+0000 7fade3fff700 1 -- 172.21.15.17:0/934683754 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a385e7d90 con 0x555a38868dd0 2023-07-14T12:01:19.013 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.027+0000 7fade939c700 1 --2- 172.21.15.17:0/934683754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a38869c20 0x555a388d0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T12:01:19.013 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.027+0000 7fade3fff700 1 --2- 172.21.15.17:0/934683754 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555a38868dd0 0x555a38874cc0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fadd00046b0 tx=0x7fadd00096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a505d037ad3d4d01 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:19.013 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.027+0000 7fadc3fff700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-07-14T12:01:19.013 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.028+0000 7fadc3fff700 1 -- 172.21.15.17:0/934683754 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd0004710 con 0x555a38868dd0 2023-07-14T12:01:19.014 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.028+0000 7fadfe6fab80 1 -- 172.21.15.17:0/934683754 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fadd80054e0 con 0x555a38868dd0 2023-07-14T12:01:19.014 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.028+0000 7fadc3fff700 1 -- 172.21.15.17:0/934683754 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fadd0004870 con 0x555a38868dd0 2023-07-14T12:01:19.014 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.028+0000 7fadc3fff700 1 -- 172.21.15.17:0/934683754 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fadd0004a30 con 0x555a38868dd0 2023-07-14T12:01:19.014 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.028+0000 7fadfe6fab80 10 client.15108 fetch_fsmap learned FSMap version 27 2023-07-14T12:01:19.014 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.028+0000 7fadfe6fab80 1 -- 172.21.15.17:0/934683754 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7faddc0b2030 con 0x555a38868dd0 2023-07-14T12:01:19.014 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.028+0000 7fadc3fff700 1 -- 172.21.15.17:0/934683754 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadd80054e0 con 0x555a38868dd0 2023-07-14T12:01:19.014 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.028+0000 7fadc3fff700 1 -- 172.21.15.17:0/934683754 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fadd0003710 con 0x555a38868dd0 2023-07-14T12:01:19.014 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.028+0000 7fadfe6fab80 10 client.15108 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T12:01:19.015 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.028+0000 7fadfe6fab80 10 client.15108 resolve_mds: resolved name 'g' to daemon mds.g 2023-07-14T12:01:19.015 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.028+0000 7fadfe6fab80 20 client.15108 populate_metadata read hostname 'smithi017' 2023-07-14T12:01:19.015 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.028+0000 7fadfe6fab80 1 --2- 172.21.15.17:0/934683754 >> [v2:172.21.15.17:6838/2976493015,v1:172.21.15.17:6839/2976493015] conn(0x555a388d60f0 0x555a388d2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.015 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.029+0000 7fadfe6fab80 4 client.15108 mds_command: new command op to 24533 tid=0[{"prefix": "status"}] 2023-07-14T12:01:19.015 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.029+0000 7fadfe6fab80 1 -- 172.21.15.17:0/934683754 --> [v2:172.21.15.17:6838/2976493015,v1:172.21.15.17:6839/2976493015] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555a38775000 con 0x555a388d60f0 2023-07-14T12:01:19.016 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.029+0000 7fade8b9b700 1 --2- 172.21.15.17:0/934683754 >> [v2:172.21.15.17:6838/2976493015,v1:172.21.15.17:6839/2976493015] conn(0x555a388d60f0 0x555a388d2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.016 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.029+0000 7fade8b9b700 1 --2- 172.21.15.17:0/934683754 >> [v2:172.21.15.17:6838/2976493015,v1:172.21.15.17:6839/2976493015] conn(0x555a388d60f0 0x555a388d2a80 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fe70ed70d6294ccb server_cookie=c4ca7e22811777d2 in_seq=0 out_seq=0 2023-07-14T12:01:19.016 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.029+0000 7fadc3fff700 10 client.15108 ms_handle_connect on v2:172.21.15.17:6838/2976493015 2023-07-14T12:01:19.017 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.030+0000 7fadc3fff700 1 -- 172.21.15.17:0/934683754 <== mds.3 v2:172.21.15.17:6838/2976493015 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x555a38775000 con 0x555a388d60f0 2023-07-14T12:01:19.017 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.030+0000 7fadc3fff700 10 client.15108 handle_command_reply: tid=0 2023-07-14T12:01:19.017 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.030+0000 7fadfe6fab80 1 client.15108 shutdown 2023-07-14T12:01:19.017 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.030+0000 7fadfe6fab80 1 -- 172.21.15.17:0/934683754 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555a38868dd0 msgr2=0x555a38874cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:19.017 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.030+0000 7fadfe6fab80 1 --2- 172.21.15.17:0/934683754 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555a38868dd0 0x555a38874cc0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fadd00046b0 tx=0x7fadd00096d0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.017 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.030+0000 7fadfe6fab80 1 -- 172.21.15.17:0/934683754 shutdown_connections 2023-07-14T12:01:19.017 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.030+0000 7fadfe6fab80 1 --2- 172.21.15.17:0/934683754 >> [v2:172.21.15.17:6838/2976493015,v1:172.21.15.17:6839/2976493015] conn(0x555a388d60f0 0x555a388d2a80 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.017 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.030+0000 7fadfe6fab80 1 --2- 172.21.15.17:0/934683754 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a38868480 0x555a38874780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.018 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.030+0000 7fadfe6fab80 1 --2- 172.21.15.17:0/934683754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555a38869c20 0x555a388d0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.018 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.030+0000 7fadfe6fab80 1 --2- 172.21.15.17:0/934683754 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555a38868dd0 0x555a38874cc0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.018 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.030+0000 7fadc3fff700 0 client.15108 ms_handle_reset on v2:172.21.15.17:6838/2976493015 2023-07-14T12:01:19.018 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.030+0000 7fadfe6fab80 1 -- 172.21.15.17:0/934683754 >> 172.21.15.17:0/934683754 conn(0x555a3885dd50 msgr2=0x555a38885c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:19.018 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.030+0000 7fadfe6fab80 1 -- 172.21.15.17:0/934683754 shutdown_connections 2023-07-14T12:01:19.018 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.030+0000 7fadfe6fab80 1 -- 172.21.15.17:0/934683754 wait complete. 2023-07-14T12:01:19.018 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.030+0000 7fadfe6fab80 20 client.15108 trim_cache size 0 max 16384 2023-07-14T12:01:19.019 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2023-07-14T12:01:19.019 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "b3c386cc-223c-11ee-9b33-001a4aab830c", 2023-07-14T12:01:19.019 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 3, 2023-07-14T12:01:19.019 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 24533, 2023-07-14T12:01:19.019 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2023-07-14T12:01:19.019 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2023-07-14T12:01:19.020 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2023-07-14T12:01:19.020 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 63.553729570999998, 2023-07-14T12:01:19.020 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 27, 2023-07-14T12:01:19.020 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 92, 2023-07-14T12:01:19.020 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 90, 2023-07-14T12:01:19.020 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 107.040904998 2023-07-14T12:01:19.020 INFO:teuthology.orchestra.run.smithi017.stdout:} 2023-07-14T12:01:19.021 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.031+0000 7fadeadff700 1 -- 172.21.15.17:0/367740119 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fadd40a1150 msgr2=0x7fadd40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:19.021 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.031+0000 7fadeadff700 1 --2- 172.21.15.17:0/367740119 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fadd40a1150 0x7fadd40a3600 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fadd00139d0 tx=0x7fadd0009b60 comp rx=0 tx=0).stop 2023-07-14T12:01:19.021 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.031+0000 7fadeadff700 1 -- 172.21.15.17:0/367740119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fade4101760 msgr2=0x7fade4072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:19.022 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.031+0000 7fadeadff700 1 --2- 172.21.15.17:0/367740119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fade4101760 0x7fade4072e40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7faddc00b070 tx=0x7faddc00bc80 comp rx=0 tx=0).stop 2023-07-14T12:01:19.022 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.031+0000 7fadeadff700 1 -- 172.21.15.17:0/367740119 shutdown_connections 2023-07-14T12:01:19.022 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.031+0000 7fadeadff700 1 --2- 172.21.15.17:0/367740119 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fadd40a1150 0x7fadd40a3600 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.022 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.031+0000 7fadeadff700 1 --2- 172.21.15.17:0/367740119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fade4101760 0x7fade4072e40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.022 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.031+0000 7fadeadff700 1 --2- 172.21.15.17:0/367740119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fade40ffbe0 0x7fade40742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.022 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.031+0000 7fadeadff700 1 --2- 172.21.15.17:0/367740119 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fade4100dd0 0x7fade4072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.022 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.031+0000 7fadeadff700 1 -- 172.21.15.17:0/367740119 >> 172.21.15.17:0/367740119 conn(0x7fade40fb200 msgr2=0x7fade40fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:19.022 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.031+0000 7fadeadff700 1 -- 172.21.15.17:0/367740119 shutdown_connections 2023-07-14T12:01:19.023 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.031+0000 7fadeadff700 1 -- 172.21.15.17:0/367740119 wait complete. 2023-07-14T12:01:19.026 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:18 smithi017 ceph-mon[126339]: pgmap v328: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 11 B/s, 0 objects/s recovering 2023-07-14T12:01:19.027 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:18 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/829115467' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T12:01:19.027 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:18 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/4290558975' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T12:01:19.029 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-07-14T12:01:19.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:18 smithi093 ceph-mon[128922]: pgmap v328: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 11 B/s, 0 objects/s recovering 2023-07-14T12:01:19.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:18 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/829115467' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T12:01:19.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:18 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/4290558975' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T12:01:19.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:18 smithi097 ceph-mon[129313]: pgmap v328: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 11 B/s, 0 objects/s recovering 2023-07-14T12:01:19.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:18 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/829115467' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T12:01:19.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:18 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/4290558975' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T12:01:19.234 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.254+0000 7fb9c65ff700 1 -- 172.21.15.17:0/2706751423 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9c0100dd0 msgr2=0x7fb9c0101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:19.405 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.254+0000 7fb9c65ff700 1 --2- 172.21.15.17:0/2706751423 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9c0100dd0 0x7fb9c0101220 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4009990 tx=0x7fb9b4009ca0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.405 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.254+0000 7fb9c65ff700 1 -- 172.21.15.17:0/2706751423 shutdown_connections 2023-07-14T12:01:19.406 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.254+0000 7fb9c65ff700 1 --2- 172.21.15.17:0/2706751423 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb9c00ffbe0 0x7fb9c00ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.406 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.254+0000 7fb9c65ff700 1 --2- 172.21.15.17:0/2706751423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9c0101760 0x7fb9c0108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.406 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.254+0000 7fb9c65ff700 1 --2- 172.21.15.17:0/2706751423 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9c0100dd0 0x7fb9c0101220 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.406 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.254+0000 7fb9c65ff700 1 -- 172.21.15.17:0/2706751423 >> 172.21.15.17:0/2706751423 conn(0x7fb9c00fb200 msgr2=0x7fb9c00fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:19.406 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.254+0000 7fb9c65ff700 1 -- 172.21.15.17:0/2706751423 shutdown_connections 2023-07-14T12:01:19.406 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.254+0000 7fb9c65ff700 1 -- 172.21.15.17:0/2706751423 wait complete. 2023-07-14T12:01:19.415 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.255+0000 7fb9c65ff700 1 Processor -- start 2023-07-14T12:01:19.415 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.255+0000 7fb9c65ff700 1 -- start start 2023-07-14T12:01:19.416 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.256+0000 7fb9c65ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9c00ffbe0 0x7fb9c00742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.416 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.256+0000 7fb9c65ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9c0100dd0 0x7fb9c0072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.416 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.256+0000 7fb9c65ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb9c0101760 0x7fb9c0072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.416 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.256+0000 7fb9c65ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c00735e0 con 0x7fb9c0100dd0 2023-07-14T12:01:19.416 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.256+0000 7fb9c65ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c0073750 con 0x7fb9c0101760 2023-07-14T12:01:19.416 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.256+0000 7fb9c65ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c019f990 con 0x7fb9c00ffbe0 2023-07-14T12:01:19.416 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.256+0000 7fb9bf7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9c0100dd0 0x7fb9c0072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.417 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.256+0000 7fb9bf7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9c0100dd0 0x7fb9c0072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53712/0 (socket says 172.21.15.17:53712) 2023-07-14T12:01:19.417 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.256+0000 7fb9bf7fe700 1 -- 172.21.15.17:0/3219733322 learned_addr learned my addr 172.21.15.17:0/3219733322 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:19.417 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.256+0000 7fb9bffff700 1 --2- 172.21.15.17:0/3219733322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9c00ffbe0 0x7fb9c00742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.417 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.256+0000 7fb9c4b9c700 1 --2- 172.21.15.17:0/3219733322 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb9c0101760 0x7fb9c0072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.417 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.256+0000 7fb9bffff700 1 -- 172.21.15.17:0/3219733322 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb9c0101760 msgr2=0x7fb9c0072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:19.417 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.256+0000 7fb9bffff700 1 --2- 172.21.15.17:0/3219733322 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb9c0101760 0x7fb9c0072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.417 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.256+0000 7fb9bffff700 1 -- 172.21.15.17:0/3219733322 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9c0100dd0 msgr2=0x7fb9c0072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:19.417 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.256+0000 7fb9bffff700 1 --2- 172.21.15.17:0/3219733322 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9c0100dd0 0x7fb9c0072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.418 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.256+0000 7fb9bffff700 1 -- 172.21.15.17:0/3219733322 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b40095f0 con 0x7fb9c00ffbe0 2023-07-14T12:01:19.418 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.256+0000 7fb9bf7fe700 1 --2- 172.21.15.17:0/3219733322 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9c0100dd0 0x7fb9c0072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T12:01:19.418 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.257+0000 7fb9bffff700 1 --2- 172.21.15.17:0/3219733322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9c00ffbe0 0x7fb9c00742c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb9b000e9c0 tx=0x7fb9b000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:19.418 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.257+0000 7fb9bd7fa700 1 -- 172.21.15.17:0/3219733322 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b0004220 con 0x7fb9c00ffbe0 2023-07-14T12:01:19.418 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.257+0000 7fb9c65ff700 1 -- 172.21.15.17:0/3219733322 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9c019fc70 con 0x7fb9c00ffbe0 2023-07-14T12:01:19.418 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.257+0000 7fb9bd7fa700 1 -- 172.21.15.17:0/3219733322 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9b0004380 con 0x7fb9c00ffbe0 2023-07-14T12:01:19.418 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.257+0000 7fb9bd7fa700 1 -- 172.21.15.17:0/3219733322 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b0003be0 con 0x7fb9c00ffbe0 2023-07-14T12:01:19.419 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.257+0000 7fb9c65ff700 1 -- 172.21.15.17:0/3219733322 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9c01a01d0 con 0x7fb9c00ffbe0 2023-07-14T12:01:19.419 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.262+0000 7fb9bd7fa700 1 -- 172.21.15.17:0/3219733322 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb9b0003d40 con 0x7fb9c00ffbe0 2023-07-14T12:01:19.419 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.263+0000 7fb9d9f01b80 1 Processor -- start 2023-07-14T12:01:19.419 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.263+0000 7fb9d9f01b80 1 -- start start 2023-07-14T12:01:19.419 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.263+0000 7fb9d9f01b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c48a213480 0x55c48a213890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.419 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.263+0000 7fb9d9f01b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c48a213dd0 0x55c48a214690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.419 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.263+0000 7fb9d9f01b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c48a214c20 0x55c48a215090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.419 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.263+0000 7fb9d9f01b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c48a118c60 con 0x55c48a214c20 2023-07-14T12:01:19.420 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.263+0000 7fb9d9f01b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c48a16b9f0 con 0x55c48a213480 2023-07-14T12:01:19.420 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.263+0000 7fb9d9f01b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c48a164e80 con 0x55c48a213dd0 2023-07-14T12:01:19.420 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.264+0000 7fb9c4b9c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c48a213480 0x55c48a213890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.420 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.264+0000 7fb9bf7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c48a214c20 0x55c48a215090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.420 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.264+0000 7fb9c4b9c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c48a213480 0x55c48a213890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.17:54242/0 (socket says 172.21.15.17:54242) 2023-07-14T12:01:19.420 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.264+0000 7fb9c4b9c700 1 -- 172.21.15.17:0/3955366894 learned_addr learned my addr 172.21.15.17:0/3955366894 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:19.420 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.264+0000 7fb9bffff700 1 --2- 172.21.15.17:0/3955366894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c48a213dd0 0x55c48a214690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.420 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.264+0000 7fb9bd7fa700 1 --2- 172.21.15.17:0/3219733322 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fb9a80a11d0 0x7fb9a80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.421 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.264+0000 7fb9bf7fe700 1 -- 172.21.15.17:0/3955366894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c48a213dd0 msgr2=0x55c48a214690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T12:01:19.421 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.264+0000 7fb9bf7fe700 1 --2- 172.21.15.17:0/3955366894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c48a213dd0 0x55c48a214690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.421 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.264+0000 7fb9bf7fe700 1 -- 172.21.15.17:0/3955366894 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c48a213480 msgr2=0x55c48a213890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T12:01:19.421 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.264+0000 7fb9bf7fe700 1 --2- 172.21.15.17:0/3955366894 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c48a213480 0x55c48a213890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.421 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.264+0000 7fb9bf7fe700 1 -- 172.21.15.17:0/3955366894 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c489f92d90 con 0x55c48a214c20 2023-07-14T12:01:19.421 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.264+0000 7fb9bd7fa700 1 -- 172.21.15.17:0/3219733322 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fb9b0014070 con 0x7fb9c00ffbe0 2023-07-14T12:01:19.421 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.264+0000 7fb9c4b9c700 1 --2- 172.21.15.17:0/3219733322 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fb9a80a11d0 0x7fb9a80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.422 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.264+0000 7fb9bf7fe700 1 --2- 172.21.15.17:0/3955366894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c48a214c20 0x55c48a215090 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb9b400b620 tx=0x7fb9b4005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b96339695d126059 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:19.422 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.265+0000 7fb9a67fc700 1 -- 172.21.15.17:0/3955366894 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b401d070 con 0x55c48a214c20 2023-07-14T12:01:19.422 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.265+0000 7fb9a67fc700 1 -- 172.21.15.17:0/3955366894 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9b401e040 con 0x55c48a214c20 2023-07-14T12:01:19.422 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.265+0000 7fb9a67fc700 1 -- 172.21.15.17:0/3955366894 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4004d70 con 0x55c48a214c20 2023-07-14T12:01:19.422 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.265+0000 7fb9d9f01b80 1 -- 172.21.15.17:0/3955366894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c48a214c20 msgr2=0x55c48a215090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:19.422 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.265+0000 7fb9c4b9c700 1 --2- 172.21.15.17:0/3219733322 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fb9a80a11d0 0x7fb9a80a3680 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8009550 tx=0x7fb9b8009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:19.422 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.265+0000 7fb9d9f01b80 1 --2- 172.21.15.17:0/3955366894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c48a214c20 0x55c48a215090 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb9b400b620 tx=0x7fb9b4005f00 comp rx=0 tx=0).stop 2023-07-14T12:01:19.422 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.266+0000 7fb9d9f01b80 1 -- 172.21.15.17:0/3955366894 shutdown_connections 2023-07-14T12:01:19.423 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.266+0000 7fb9d9f01b80 1 --2- 172.21.15.17:0/3955366894 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c48a213480 0x55c48a213890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.423 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.266+0000 7fb9d9f01b80 1 --2- 172.21.15.17:0/3955366894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c48a214c20 0x55c48a215090 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.423 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.266+0000 7fb9d9f01b80 1 --2- 172.21.15.17:0/3955366894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c48a213dd0 0x55c48a214690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.423 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.266+0000 7fb9d9f01b80 1 -- 172.21.15.17:0/3955366894 >> 172.21.15.17:0/3955366894 conn(0x55c48a208d50 msgr2=0x55c48a20e180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:19.423 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.266+0000 7fb9d9f01b80 1 -- 172.21.15.17:0/3955366894 shutdown_connections 2023-07-14T12:01:19.423 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.266+0000 7fb9d9f01b80 1 -- 172.21.15.17:0/3955366894 wait complete. 2023-07-14T12:01:19.423 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.266+0000 7fb9d9f01b80 1 -- start start 2023-07-14T12:01:19.424 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.266+0000 7fb9d9f01b80 1 Processor -- start 2023-07-14T12:01:19.424 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.267+0000 7fb9d9f01b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c48a213480 0x55c48a21d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.424 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.267+0000 7fb9d9f01b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c48a213dd0 0x55c48a21d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.424 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.267+0000 7fb9d9f01b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c48a214c20 0x55c48a2245b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.424 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.267+0000 7fb9d9f01b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c48a104a70 con 0x55c48a214c20 2023-07-14T12:01:19.424 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.267+0000 7fb9d9f01b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c48a104770 con 0x55c48a213480 2023-07-14T12:01:19.424 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.267+0000 7fb9d9f01b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c489fe88d0 con 0x55c48a213dd0 2023-07-14T12:01:19.424 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.267+0000 7fb9d9f01b80 10 client.0 _ll_register_callbacks cb 0x55c48a102710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T12:01:19.425 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.267+0000 7fb9c4b9c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c48a214c20 0x55c48a2245b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.425 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.267+0000 7fb9c4b9c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c48a214c20 0x55c48a2245b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53726/0 (socket says 172.21.15.17:53726) 2023-07-14T12:01:19.425 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.267+0000 7fb9c4b9c700 1 -- 172.21.15.17:0/3737179894 learned_addr learned my addr 172.21.15.17:0/3737179894 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:19.425 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.267+0000 7fb9bf7fe700 1 --2- 172.21.15.17:0/3737179894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c48a213dd0 0x55c48a21d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.425 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.267+0000 7fb9bffff700 1 --2- 172.21.15.17:0/3737179894 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c48a213480 0x55c48a21d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.425 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.268+0000 7fb9bf7fe700 1 -- 172.21.15.17:0/3737179894 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c48a213480 msgr2=0x55c48a21d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T12:01:19.425 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.268+0000 7fb9bf7fe700 1 --2- 172.21.15.17:0/3737179894 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c48a213480 0x55c48a21d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.425 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.268+0000 7fb9bf7fe700 1 -- 172.21.15.17:0/3737179894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c48a214c20 msgr2=0x55c48a2245b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T12:01:19.426 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.268+0000 7fb9bf7fe700 1 --2- 172.21.15.17:0/3737179894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c48a214c20 0x55c48a2245b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.426 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.268+0000 7fb9bf7fe700 1 -- 172.21.15.17:0/3737179894 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9c0108950 con 0x55c48a213dd0 2023-07-14T12:01:19.426 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.268+0000 7fb9bf7fe700 1 --2- 172.21.15.17:0/3737179894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c48a213dd0 0x55c48a21d570 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb9b400b620 tx=0x7fb9b4005a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d345b8b23fae244d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:19.426 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.268+0000 7fb99f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-07-14T12:01:19.426 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.268+0000 7fb99f7fe700 1 -- 172.21.15.17:0/3737179894 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b401d030 con 0x55c48a213dd0 2023-07-14T12:01:19.426 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.268+0000 7fb9d9f01b80 1 -- 172.21.15.17:0/3737179894 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c48a0be4a0 con 0x55c48a213dd0 2023-07-14T12:01:19.426 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.268+0000 7fb99f7fe700 1 -- 172.21.15.17:0/3737179894 <== mon.1 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fb9b4022bc0 con 0x55c48a213dd0 2023-07-14T12:01:19.427 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.269+0000 7fb99f7fe700 1 -- 172.21.15.17:0/3737179894 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9b4022d80 con 0x55c48a213dd0 2023-07-14T12:01:19.427 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.269+0000 7fb9d9f01b80 10 client.25144 fetch_fsmap learned FSMap version 27 2023-07-14T12:01:19.427 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.269+0000 7fb9d9f01b80 1 -- 172.21.15.17:0/3737179894 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c489f7bc10 con 0x55c48a213dd0 2023-07-14T12:01:19.427 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.269+0000 7fb99f7fe700 1 -- 172.21.15.17:0/3737179894 <== mon.1 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c48a0be4a0 con 0x55c48a213dd0 2023-07-14T12:01:19.427 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.269+0000 7fb99f7fe700 1 -- 172.21.15.17:0/3737179894 <== mon.1 v2:172.21.15.97:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb9b402c570 con 0x55c48a213dd0 2023-07-14T12:01:19.427 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.269+0000 7fb9d9f01b80 10 client.25144 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T12:01:19.427 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.269+0000 7fb9d9f01b80 10 client.25144 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-14T12:01:19.427 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.269+0000 7fb9d9f01b80 20 client.25144 populate_metadata read hostname 'smithi017' 2023-07-14T12:01:19.428 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.269+0000 7fb9d9f01b80 1 --2- 172.21.15.17:0/3737179894 >> [v2:172.21.15.97:6838/3267815331,v1:172.21.15.97:6839/3267815331] conn(0x55c48a227ea0 0x55c48a2274e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.428 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.269+0000 7fb9d9f01b80 4 client.25144 mds_command: new command op to 24541 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T12:01:19.428 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.269+0000 7fb9d9f01b80 1 -- 172.21.15.17:0/3737179894 --> [v2:172.21.15.97:6838/3267815331,v1:172.21.15.97:6839/3267815331] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c48a15de60 con 0x55c48a227ea0 2023-07-14T12:01:19.428 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.270+0000 7fb9bffff700 1 --2- 172.21.15.17:0/3737179894 >> [v2:172.21.15.97:6838/3267815331,v1:172.21.15.97:6839/3267815331] conn(0x55c48a227ea0 0x55c48a2274e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.428 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.270+0000 7fb9bffff700 1 --2- 172.21.15.17:0/3737179894 >> [v2:172.21.15.97:6838/3267815331,v1:172.21.15.97:6839/3267815331] conn(0x55c48a227ea0 0x55c48a2274e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aebddef7abfdda79 server_cookie=95f1a410fed08543 in_seq=0 out_seq=0 2023-07-14T12:01:19.428 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.270+0000 7fb99f7fe700 10 client.25144 ms_handle_connect on v2:172.21.15.97:6838/3267815331 2023-07-14T12:01:19.428 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.271+0000 7fb99f7fe700 1 -- 172.21.15.17:0/3737179894 <== mds.2 v2:172.21.15.97:6838/3267815331 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c48a15de60 con 0x55c48a227ea0 2023-07-14T12:01:19.429 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.271+0000 7fb99f7fe700 10 client.25144 handle_command_reply: tid=0 2023-07-14T12:01:19.429 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.271+0000 7fb9d9f01b80 1 client.25144 shutdown 2023-07-14T12:01:19.429 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.271+0000 7fb9d9f01b80 1 -- 172.21.15.17:0/3737179894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c48a213dd0 msgr2=0x55c48a21d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:19.429 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.271+0000 7fb9d9f01b80 1 --2- 172.21.15.17:0/3737179894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c48a213dd0 0x55c48a21d570 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb9b400b620 tx=0x7fb9b4005a40 comp rx=0 tx=0).stop 2023-07-14T12:01:19.429 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.271+0000 7fb9d9f01b80 1 -- 172.21.15.17:0/3737179894 shutdown_connections 2023-07-14T12:01:19.429 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.271+0000 7fb9d9f01b80 1 --2- 172.21.15.17:0/3737179894 >> [v2:172.21.15.97:6838/3267815331,v1:172.21.15.97:6839/3267815331] conn(0x55c48a227ea0 0x55c48a2274e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.429 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.271+0000 7fb9d9f01b80 1 --2- 172.21.15.17:0/3737179894 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c48a213480 0x55c48a21d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.429 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.271+0000 7fb99f7fe700 0 client.25144 ms_handle_reset on v2:172.21.15.97:6838/3267815331 2023-07-14T12:01:19.430 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.271+0000 7fb9d9f01b80 1 --2- 172.21.15.17:0/3737179894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c48a214c20 0x55c48a2245b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.430 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.271+0000 7fb9d9f01b80 1 --2- 172.21.15.17:0/3737179894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c48a213dd0 0x55c48a21d570 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.430 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.271+0000 7fb9d9f01b80 1 -- 172.21.15.17:0/3737179894 >> 172.21.15.17:0/3737179894 conn(0x55c48a208d50 msgr2=0x55c48a20d860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:19.430 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.271+0000 7fb9d9f01b80 1 -- 172.21.15.17:0/3737179894 shutdown_connections 2023-07-14T12:01:19.430 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.272+0000 7fb9d9f01b80 1 -- 172.21.15.17:0/3737179894 wait complete. 2023-07-14T12:01:19.430 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.272+0000 7fb9d9f01b80 20 client.25144 trim_cache size 0 max 16384 2023-07-14T12:01:19.430 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.290+0000 7fb9d9f01b80 1 Processor -- start 2023-07-14T12:01:19.431 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.290+0000 7fb9d9f01b80 1 -- start start 2023-07-14T12:01:19.431 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.290+0000 7fb9d9f01b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c48a213480 0x55c48a2274e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.431 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.290+0000 7fb9d9f01b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c48a213dd0 0x55c48a2244d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.431 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.290+0000 7fb9d9f01b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c48a214c20 0x55c48a20d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.431 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.291+0000 7fb9d9f01b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c48a10e470 con 0x55c48a213480 2023-07-14T12:01:19.431 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.291+0000 7fb9d9f01b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c48a133790 con 0x55c48a214c20 2023-07-14T12:01:19.431 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.291+0000 7fb9d9f01b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c489ff4de0 con 0x55c48a213dd0 2023-07-14T12:01:19.431 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.291+0000 7fb9bffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c48a213480 0x55c48a2274e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.432 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.291+0000 7fb9bffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c48a213480 0x55c48a2274e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53736/0 (socket says 172.21.15.17:53736) 2023-07-14T12:01:19.432 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.291+0000 7fb9bffff700 1 -- 172.21.15.17:0/1754461869 learned_addr learned my addr 172.21.15.17:0/1754461869 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:19.432 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.291+0000 7fb9bf7fe700 1 --2- 172.21.15.17:0/1754461869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c48a213dd0 0x55c48a2244d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.432 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.291+0000 7fb9c4b9c700 1 --2- 172.21.15.17:0/1754461869 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c48a214c20 0x55c48a20d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.432 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.291+0000 7fb9bffff700 1 -- 172.21.15.17:0/1754461869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c48a213dd0 msgr2=0x55c48a2244d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T12:01:19.432 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.291+0000 7fb9bffff700 1 --2- 172.21.15.17:0/1754461869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c48a213dd0 0x55c48a2244d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.432 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.291+0000 7fb9bffff700 1 -- 172.21.15.17:0/1754461869 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c48a214c20 msgr2=0x55c48a20d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T12:01:19.433 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.291+0000 7fb9bffff700 1 --2- 172.21.15.17:0/1754461869 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c48a214c20 0x55c48a20d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.433 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.291+0000 7fb9bffff700 1 -- 172.21.15.17:0/1754461869 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c489fb9200 con 0x55c48a213480 2023-07-14T12:01:19.433 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.292+0000 7fb9bffff700 1 --2- 172.21.15.17:0/1754461869 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c48a213480 0x55c48a2274e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb9b000f010 tx=0x7fb9b0003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d3d42e88cfdd81 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:19.433 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.292+0000 7fb9a6ffd700 1 -- 172.21.15.17:0/1754461869 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b0099890 con 0x55c48a213480 2023-07-14T12:01:19.433 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.292+0000 7fb9a6ffd700 1 -- 172.21.15.17:0/1754461869 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9b00044f0 con 0x55c48a213480 2023-07-14T12:01:19.433 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.292+0000 7fb9a6ffd700 1 -- 172.21.15.17:0/1754461869 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b00ca730 con 0x55c48a213480 2023-07-14T12:01:19.433 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.292+0000 7fb9d9f01b80 1 -- 172.21.15.17:0/1754461869 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c48a213480 msgr2=0x55c48a2274e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:19.433 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.292+0000 7fb9d9f01b80 1 --2- 172.21.15.17:0/1754461869 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c48a213480 0x55c48a2274e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb9b000f010 tx=0x7fb9b0003680 comp rx=0 tx=0).stop 2023-07-14T12:01:19.434 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.292+0000 7fb9d9f01b80 1 -- 172.21.15.17:0/1754461869 shutdown_connections 2023-07-14T12:01:19.434 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.292+0000 7fb9d9f01b80 1 --2- 172.21.15.17:0/1754461869 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c48a214c20 0x55c48a20d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.434 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.292+0000 7fb9d9f01b80 1 --2- 172.21.15.17:0/1754461869 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c48a213480 0x55c48a2274e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.434 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.292+0000 7fb9d9f01b80 1 --2- 172.21.15.17:0/1754461869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c48a213dd0 0x55c48a2244d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.434 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.292+0000 7fb9d9f01b80 1 -- 172.21.15.17:0/1754461869 >> 172.21.15.17:0/1754461869 conn(0x55c48a208d50 msgr2=0x55c48a21bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:19.434 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.292+0000 7fb9d9f01b80 1 -- 172.21.15.17:0/1754461869 shutdown_connections 2023-07-14T12:01:19.434 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.292+0000 7fb9d9f01b80 1 -- 172.21.15.17:0/1754461869 wait complete. 2023-07-14T12:01:19.435 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.293+0000 7fb9d9f01b80 1 -- start start 2023-07-14T12:01:19.435 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.293+0000 7fb9d9f01b80 1 Processor -- start 2023-07-14T12:01:19.435 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.294+0000 7fb9d9f01b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c48a213480 0x55c48a21f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.435 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.294+0000 7fb9d9f01b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c48a213dd0 0x55c48a21fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.435 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.294+0000 7fb9d9f01b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c48a214c20 0x55c48a222d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.435 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.294+0000 7fb9d9f01b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c48a2232c0 con 0x55c48a214c20 2023-07-14T12:01:19.435 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.294+0000 7fb9d9f01b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c48a223430 con 0x55c48a213dd0 2023-07-14T12:01:19.435 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.294+0000 7fb9d9f01b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c48a2235f0 con 0x55c48a213480 2023-07-14T12:01:19.436 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.294+0000 7fb9d9f01b80 10 client.0 _ll_register_callbacks cb 0x7fb9b8009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T12:01:19.436 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.294+0000 7fb9c4b9c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c48a214c20 0x55c48a222d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.436 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.294+0000 7fb9c4b9c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c48a214c20 0x55c48a222d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53742/0 (socket says 172.21.15.17:53742) 2023-07-14T12:01:19.436 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.294+0000 7fb9bffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c48a213480 0x55c48a21f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.436 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.294+0000 7fb9c4b9c700 1 -- 172.21.15.17:0/2681330818 learned_addr learned my addr 172.21.15.17:0/2681330818 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:19.436 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.294+0000 7fb9bf7fe700 1 --2- 172.21.15.17:0/2681330818 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c48a213dd0 0x55c48a21fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.436 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.295+0000 7fb9bffff700 1 -- 172.21.15.17:0/2681330818 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c48a213dd0 msgr2=0x55c48a21fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T12:01:19.437 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.295+0000 7fb9bffff700 1 --2- 172.21.15.17:0/2681330818 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c48a213dd0 0x55c48a21fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.437 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.295+0000 7fb9bffff700 1 -- 172.21.15.17:0/2681330818 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c48a214c20 msgr2=0x55c48a222d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T12:01:19.437 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.295+0000 7fb9bffff700 1 --2- 172.21.15.17:0/2681330818 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c48a214c20 0x55c48a222d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.437 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.295+0000 7fb9bffff700 1 -- 172.21.15.17:0/2681330818 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9c01a01d0 con 0x55c48a213480 2023-07-14T12:01:19.437 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.295+0000 7fb9bf7fe700 1 --2- 172.21.15.17:0/2681330818 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c48a213dd0 0x55c48a21fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T12:01:19.437 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.295+0000 7fb9c4b9c700 1 --2- 172.21.15.17:0/2681330818 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c48a214c20 0x55c48a222d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T12:01:19.437 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.295+0000 7fb9bffff700 1 --2- 172.21.15.17:0/2681330818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c48a213480 0x55c48a21f820 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x55c48a104ae0 tx=0x7fb9b0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7848bdadfe447bd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:19.437 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.295+0000 7fb99f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-07-14T12:01:19.438 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.295+0000 7fb99f7fe700 1 -- 172.21.15.17:0/2681330818 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b00d0040 con 0x55c48a213480 2023-07-14T12:01:19.438 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.295+0000 7fb99f7fe700 1 -- 172.21.15.17:0/2681330818 <== mon.1 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fb9b0099f30 con 0x55c48a213480 2023-07-14T12:01:19.438 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.295+0000 7fb9d9f01b80 1 -- 172.21.15.17:0/2681330818 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9b401e040 con 0x55c48a213480 2023-07-14T12:01:19.438 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.295+0000 7fb99f7fe700 1 -- 172.21.15.17:0/2681330818 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9b0016330 con 0x55c48a213480 2023-07-14T12:01:19.438 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.295+0000 7fb9d9f01b80 10 client.25150 fetch_fsmap learned FSMap version 27 2023-07-14T12:01:19.438 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.295+0000 7fb9d9f01b80 1 -- 172.21.15.17:0/2681330818 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c489f91260 con 0x55c48a213480 2023-07-14T12:01:19.438 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.295+0000 7fb99f7fe700 1 -- 172.21.15.17:0/2681330818 <== mon.1 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9b00c8080 con 0x55c48a213480 2023-07-14T12:01:19.439 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.296+0000 7fb99f7fe700 1 -- 172.21.15.17:0/2681330818 <== mon.1 v2:172.21.15.97:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb9b00d2070 con 0x55c48a213480 2023-07-14T12:01:19.439 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.296+0000 7fb9d9f01b80 10 client.25150 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T12:01:19.439 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.296+0000 7fb9d9f01b80 10 client.25150 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-14T12:01:19.439 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.296+0000 7fb9d9f01b80 20 client.25150 populate_metadata read hostname 'smithi017' 2023-07-14T12:01:19.439 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.296+0000 7fb9d9f01b80 1 --2- 172.21.15.17:0/2681330818 >> [v2:172.21.15.97:6838/3267815331,v1:172.21.15.97:6839/3267815331] conn(0x55c48a281180 0x55c48a27dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.439 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.296+0000 7fb9d9f01b80 4 client.25150 mds_command: new command op to 24541 tid=0[{"prefix": "status"}] 2023-07-14T12:01:19.439 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.296+0000 7fb9d9f01b80 1 -- 172.21.15.17:0/2681330818 --> [v2:172.21.15.97:6838/3267815331,v1:172.21.15.97:6839/3267815331] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c48a120000 con 0x55c48a281180 2023-07-14T12:01:19.439 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.296+0000 7fb9bf7fe700 1 --2- 172.21.15.17:0/2681330818 >> [v2:172.21.15.97:6838/3267815331,v1:172.21.15.97:6839/3267815331] conn(0x55c48a281180 0x55c48a27dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.440 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.297+0000 7fb9bf7fe700 1 --2- 172.21.15.17:0/2681330818 >> [v2:172.21.15.97:6838/3267815331,v1:172.21.15.97:6839/3267815331] conn(0x55c48a281180 0x55c48a27dd50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c254231fba4e9d6d server_cookie=e005c05d331ce9cf in_seq=0 out_seq=0 2023-07-14T12:01:19.440 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.297+0000 7fb99f7fe700 10 client.25150 ms_handle_connect on v2:172.21.15.97:6838/3267815331 2023-07-14T12:01:19.440 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.297+0000 7fb99f7fe700 1 -- 172.21.15.17:0/2681330818 <== mds.2 v2:172.21.15.97:6838/3267815331 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55c48a120000 con 0x55c48a281180 2023-07-14T12:01:19.440 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.297+0000 7fb99f7fe700 10 client.25150 handle_command_reply: tid=0 2023-07-14T12:01:19.440 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.297+0000 7fb9d9f01b80 1 client.25150 shutdown 2023-07-14T12:01:19.440 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.297+0000 7fb9d9f01b80 1 -- 172.21.15.17:0/2681330818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c48a213480 msgr2=0x55c48a21f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:19.440 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.297+0000 7fb9d9f01b80 1 --2- 172.21.15.17:0/2681330818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c48a213480 0x55c48a21f820 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x55c48a104ae0 tx=0x7fb9b0004750 comp rx=0 tx=0).stop 2023-07-14T12:01:19.440 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.297+0000 7fb9d9f01b80 1 -- 172.21.15.17:0/2681330818 shutdown_connections 2023-07-14T12:01:19.441 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.297+0000 7fb9d9f01b80 1 --2- 172.21.15.17:0/2681330818 >> [v2:172.21.15.97:6838/3267815331,v1:172.21.15.97:6839/3267815331] conn(0x55c48a281180 0x55c48a27dd50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.441 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.297+0000 7fb9d9f01b80 1 --2- 172.21.15.17:0/2681330818 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c48a214c20 0x55c48a222d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.441 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.297+0000 7fb99f7fe700 0 client.25150 ms_handle_reset on v2:172.21.15.97:6838/3267815331 2023-07-14T12:01:19.441 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.297+0000 7fb9d9f01b80 1 --2- 172.21.15.17:0/2681330818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c48a213480 0x55c48a21f820 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.441 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.297+0000 7fb9d9f01b80 1 --2- 172.21.15.17:0/2681330818 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c48a213dd0 0x55c48a21fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.441 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.297+0000 7fb9d9f01b80 1 -- 172.21.15.17:0/2681330818 >> 172.21.15.17:0/2681330818 conn(0x55c48a208d50 msgr2=0x55c48a2274e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:19.441 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.297+0000 7fb9d9f01b80 1 -- 172.21.15.17:0/2681330818 shutdown_connections 2023-07-14T12:01:19.442 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.297+0000 7fb9d9f01b80 1 -- 172.21.15.17:0/2681330818 wait complete. 2023-07-14T12:01:19.442 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.297+0000 7fb9d9f01b80 20 client.25150 trim_cache size 0 max 16384 2023-07-14T12:01:19.442 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2023-07-14T12:01:19.442 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "b3c386cc-223c-11ee-9b33-001a4aab830c", 2023-07-14T12:01:19.442 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 2, 2023-07-14T12:01:19.443 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 24541, 2023-07-14T12:01:19.443 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2023-07-14T12:01:19.443 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2023-07-14T12:01:19.443 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2023-07-14T12:01:19.443 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 64.829226352999996, 2023-07-14T12:01:19.443 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 27, 2023-07-14T12:01:19.443 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 92, 2023-07-14T12:01:19.443 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 90, 2023-07-14T12:01:19.444 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 110.49481340299999 2023-07-14T12:01:19.444 INFO:teuthology.orchestra.run.smithi017.stdout:} 2023-07-14T12:01:19.445 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.298+0000 7fb9c65ff700 1 -- 172.21.15.17:0/3219733322 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fb9a80a11d0 msgr2=0x7fb9a80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:19.445 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.298+0000 7fb9c65ff700 1 --2- 172.21.15.17:0/3219733322 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fb9a80a11d0 0x7fb9a80a3680 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8009550 tx=0x7fb9b8009040 comp rx=0 tx=0).stop 2023-07-14T12:01:19.445 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.298+0000 7fb9c65ff700 1 -- 172.21.15.17:0/3219733322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9c00ffbe0 msgr2=0x7fb9c00742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:19.445 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.298+0000 7fb9c65ff700 1 --2- 172.21.15.17:0/3219733322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9c00ffbe0 0x7fb9c00742c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb9b000e9c0 tx=0x7fb9b000ecd0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.445 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.298+0000 7fb9c65ff700 1 -- 172.21.15.17:0/3219733322 shutdown_connections 2023-07-14T12:01:19.446 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.298+0000 7fb9c65ff700 1 --2- 172.21.15.17:0/3219733322 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fb9a80a11d0 0x7fb9a80a3680 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.446 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.298+0000 7fb9c65ff700 1 --2- 172.21.15.17:0/3219733322 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb9c0101760 0x7fb9c0072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.446 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.298+0000 7fb9c65ff700 1 --2- 172.21.15.17:0/3219733322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9c00ffbe0 0x7fb9c00742c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.446 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.298+0000 7fb9c65ff700 1 --2- 172.21.15.17:0/3219733322 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9c0100dd0 0x7fb9c0072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.446 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.298+0000 7fb9c65ff700 1 -- 172.21.15.17:0/3219733322 >> 172.21.15.17:0/3219733322 conn(0x7fb9c00fb200 msgr2=0x7fb9c0104260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:19.446 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.298+0000 7fb9c65ff700 1 -- 172.21.15.17:0/3219733322 shutdown_connections 2023-07-14T12:01:19.446 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.298+0000 7fb9c65ff700 1 -- 172.21.15.17:0/3219733322 wait complete. 2023-07-14T12:01:19.447 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-07-14T12:01:19.655 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.675+0000 7faf9bfff700 1 -- 172.21.15.17:0/2144739293 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf9c068500 msgr2=0x7faf9c1024b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:19.655 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.675+0000 7faf9bfff700 1 --2- 172.21.15.17:0/2144739293 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf9c068500 0x7faf9c1024b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7faf8c009990 tx=0x7faf8c009ca0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.656 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.675+0000 7faf9bfff700 1 -- 172.21.15.17:0/2144739293 shutdown_connections 2023-07-14T12:01:19.656 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.675+0000 7faf9bfff700 1 --2- 172.21.15.17:0/2144739293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf9c1054e0 0x7faf9c107990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.656 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.675+0000 7faf9bfff700 1 --2- 172.21.15.17:0/2144739293 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf9c068500 0x7faf9c1024b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.656 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.675+0000 7faf9bfff700 1 --2- 172.21.15.17:0/2144739293 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faf9c1029f0 0x7faf9c104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.656 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.675+0000 7faf9bfff700 1 -- 172.21.15.17:0/2144739293 >> 172.21.15.17:0/2144739293 conn(0x7faf9c0f9de0 msgr2=0x7faf9c0fc250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:19.656 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.675+0000 7faf9bfff700 1 -- 172.21.15.17:0/2144739293 shutdown_connections 2023-07-14T12:01:19.656 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.675+0000 7faf9bfff700 1 -- 172.21.15.17:0/2144739293 wait complete. 2023-07-14T12:01:19.658 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.676+0000 7faf9bfff700 1 Processor -- start 2023-07-14T12:01:19.658 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.676+0000 7faf9bfff700 1 -- start start 2023-07-14T12:01:19.658 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.676+0000 7faf9bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf9c068500 0x7faf9c1978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.659 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.676+0000 7faf9bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faf9c1029f0 0x7faf9c197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.659 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.677+0000 7faf9bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf9c1054e0 0x7faf9c19ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.659 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.677+0000 7faf9bfff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c19f450 con 0x7faf9c1054e0 2023-07-14T12:01:19.659 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.677+0000 7faf9bfff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c19f590 con 0x7faf9c1029f0 2023-07-14T12:01:19.659 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.677+0000 7faf9bfff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c19f720 con 0x7faf9c068500 2023-07-14T12:01:19.659 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.677+0000 7faf9affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf9c068500 0x7faf9c1978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.659 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.677+0000 7faf9affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf9c068500 0x7faf9c1978d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.17:54354/0 (socket says 172.21.15.17:54354) 2023-07-14T12:01:19.659 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.677+0000 7faf9affd700 1 -- 172.21.15.17:0/2528732744 learned_addr learned my addr 172.21.15.17:0/2528732744 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:19.660 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.677+0000 7faf9b7fe700 1 --2- 172.21.15.17:0/2528732744 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf9c1054e0 0x7faf9c19ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.660 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.677+0000 7faf93fff700 1 --2- 172.21.15.17:0/2528732744 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faf9c1029f0 0x7faf9c197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.660 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.677+0000 7faf9affd700 1 -- 172.21.15.17:0/2528732744 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faf9c1029f0 msgr2=0x7faf9c197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:19.660 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.677+0000 7faf9affd700 1 --2- 172.21.15.17:0/2528732744 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faf9c1029f0 0x7faf9c197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.660 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.677+0000 7faf9affd700 1 -- 172.21.15.17:0/2528732744 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf9c1054e0 msgr2=0x7faf9c19ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:19.660 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.677+0000 7faf9affd700 1 --2- 172.21.15.17:0/2528732744 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf9c1054e0 0x7faf9c19ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.660 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.677+0000 7faf9affd700 1 -- 172.21.15.17:0/2528732744 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf8c0095f0 con 0x7faf9c068500 2023-07-14T12:01:19.661 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.677+0000 7faf9b7fe700 1 --2- 172.21.15.17:0/2528732744 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf9c1054e0 0x7faf9c19ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T12:01:19.661 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.678+0000 7faf9affd700 1 --2- 172.21.15.17:0/2528732744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf9c068500 0x7faf9c1978d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7faf8c005720 tx=0x7faf8c009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:19.661 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.678+0000 7faf98ff9700 1 -- 172.21.15.17:0/2528732744 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c01d070 con 0x7faf9c068500 2023-07-14T12:01:19.661 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.678+0000 7faf9bfff700 1 -- 172.21.15.17:0/2528732744 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf9c19f9a0 con 0x7faf9c068500 2023-07-14T12:01:19.661 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.678+0000 7faf98ff9700 1 -- 172.21.15.17:0/2528732744 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf8c022470 con 0x7faf9c068500 2023-07-14T12:01:19.661 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.678+0000 7faf98ff9700 1 -- 172.21.15.17:0/2528732744 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c017650 con 0x7faf9c068500 2023-07-14T12:01:19.661 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.678+0000 7faf9bfff700 1 -- 172.21.15.17:0/2528732744 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf9c19fdc0 con 0x7faf9c068500 2023-07-14T12:01:19.662 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.681+0000 7faf98ff9700 1 -- 172.21.15.17:0/2528732744 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7faf8c0225e0 con 0x7faf9c068500 2023-07-14T12:01:19.662 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.682+0000 7fafb4f21b80 1 Processor -- start 2023-07-14T12:01:19.663 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.682+0000 7fafb4f21b80 1 -- start start 2023-07-14T12:01:19.663 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.683+0000 7fafb4f21b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f8d7086d0 0x563f8d708ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.663 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.683+0000 7fafb4f21b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f8d709020 0x563f8d709850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.664 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.683+0000 7fafb4f21b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563f8d709de0 0x563f8d70a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.664 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.683+0000 7fafb4f21b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f8d60dc60 con 0x563f8d7086d0 2023-07-14T12:01:19.664 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.683+0000 7fafb4f21b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f8d6609f0 con 0x563f8d709020 2023-07-14T12:01:19.664 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.683+0000 7fafb4f21b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f8d659e80 con 0x563f8d709de0 2023-07-14T12:01:19.664 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.683+0000 7faf9b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f8d7086d0 0x563f8d708ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.664 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.683+0000 7faf98ff9700 1 --2- 172.21.15.17:0/2528732744 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7faf880a1150 0x7faf880a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.664 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.683+0000 7faf9b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f8d7086d0 0x563f8d708ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53776/0 (socket says 172.21.15.17:53776) 2023-07-14T12:01:19.664 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.683+0000 7faf9b7fe700 1 -- 172.21.15.17:0/779645252 learned_addr learned my addr 172.21.15.17:0/779645252 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:19.665 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.683+0000 7faf9affd700 1 --2- 172.21.15.17:0/779645252 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f8d709020 0x563f8d709850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.665 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.683+0000 7faf93fff700 1 --2- 172.21.15.17:0/779645252 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563f8d709de0 0x563f8d70a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.665 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.683+0000 7faf98ff9700 1 -- 172.21.15.17:0/2528732744 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7faf8c0ce0a0 con 0x7faf9c068500 2023-07-14T12:01:19.667 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.683+0000 7faf9b7fe700 1 --2- 172.21.15.17:0/2528732744 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7faf880a1150 0x7faf880a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.667 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.684+0000 7faf93fff700 1 -- 172.21.15.17:0/779645252 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f8d709020 msgr2=0x563f8d709850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T12:01:19.667 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.684+0000 7faf93fff700 1 --2- 172.21.15.17:0/779645252 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f8d709020 0x563f8d709850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.668 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.684+0000 7faf93fff700 1 -- 172.21.15.17:0/779645252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f8d7086d0 msgr2=0x563f8d708ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T12:01:19.668 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.684+0000 7faf93fff700 1 --2- 172.21.15.17:0/779645252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f8d7086d0 0x563f8d708ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.668 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.684+0000 7faf93fff700 1 -- 172.21.15.17:0/779645252 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f8d487d90 con 0x563f8d709de0 2023-07-14T12:01:19.668 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.684+0000 7faf9affd700 1 --2- 172.21.15.17:0/779645252 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f8d709020 0x563f8d709850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T12:01:19.668 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.684+0000 7faf93fff700 1 --2- 172.21.15.17:0/779645252 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563f8d709de0 0x563f8d70a250 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7faf84009cb0 tx=0x7faf84011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9abac1524b1dc6c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:19.668 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.684+0000 7faf9b7fe700 1 --2- 172.21.15.17:0/2528732744 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7faf880a1150 0x7faf880a3600 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7faf9400ef00 tx=0x7faf94004d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:19.668 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.684+0000 7faf917fa700 1 -- 172.21.15.17:0/779645252 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf84011c60 con 0x563f8d709de0 2023-07-14T12:01:19.669 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.684+0000 7faf917fa700 1 -- 172.21.15.17:0/779645252 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf84004510 con 0x563f8d709de0 2023-07-14T12:01:19.669 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.684+0000 7faf917fa700 1 -- 172.21.15.17:0/779645252 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf84005280 con 0x563f8d709de0 2023-07-14T12:01:19.669 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.685+0000 7fafb4f21b80 1 -- 172.21.15.17:0/779645252 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563f8d709de0 msgr2=0x563f8d70a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:19.669 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.685+0000 7fafb4f21b80 1 --2- 172.21.15.17:0/779645252 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563f8d709de0 0x563f8d70a250 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7faf84009cb0 tx=0x7faf84011440 comp rx=0 tx=0).stop 2023-07-14T12:01:19.669 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.685+0000 7fafb4f21b80 1 -- 172.21.15.17:0/779645252 shutdown_connections 2023-07-14T12:01:19.669 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.685+0000 7fafb4f21b80 1 --2- 172.21.15.17:0/779645252 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563f8d709de0 0x563f8d70a250 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.669 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.685+0000 7fafb4f21b80 1 --2- 172.21.15.17:0/779645252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f8d7086d0 0x563f8d708ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.685+0000 7fafb4f21b80 1 --2- 172.21.15.17:0/779645252 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f8d709020 0x563f8d709850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.685+0000 7fafb4f21b80 1 -- 172.21.15.17:0/779645252 >> 172.21.15.17:0/779645252 conn(0x563f8d6fdfa0 msgr2=0x563f8d703410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:19.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.685+0000 7fafb4f21b80 1 -- 172.21.15.17:0/779645252 shutdown_connections 2023-07-14T12:01:19.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.685+0000 7fafb4f21b80 1 -- 172.21.15.17:0/779645252 wait complete. 2023-07-14T12:01:19.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.685+0000 7fafb4f21b80 1 -- start start 2023-07-14T12:01:19.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.686+0000 7fafb4f21b80 1 Processor -- start 2023-07-14T12:01:19.670 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.686+0000 7fafb4f21b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563f8d7086d0 0x563f8d7122c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.671 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.686+0000 7fafb4f21b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f8d709020 0x563f8d712800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.671 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.686+0000 7fafb4f21b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f8d709de0 0x563f8d719840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.671 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.686+0000 7fafb4f21b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f8d5f9a70 con 0x563f8d709020 2023-07-14T12:01:19.671 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.686+0000 7fafb4f21b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f8d5f9770 con 0x563f8d709de0 2023-07-14T12:01:19.671 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.686+0000 7fafb4f21b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f8d4dd8d0 con 0x563f8d7086d0 2023-07-14T12:01:19.671 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.686+0000 7fafb4f21b80 10 client.0 _ll_register_callbacks cb 0x563f8d5f7710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T12:01:19.671 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.686+0000 7faf93fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f8d709020 0x563f8d712800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.671 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.687+0000 7faf93fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f8d709020 0x563f8d712800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53784/0 (socket says 172.21.15.17:53784) 2023-07-14T12:01:19.672 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.687+0000 7faf9affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563f8d7086d0 0x563f8d7122c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.672 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.687+0000 7faf93fff700 1 -- 172.21.15.17:0/2222301923 learned_addr learned my addr 172.21.15.17:0/2222301923 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:19.672 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.687+0000 7faf9b7fe700 1 --2- 172.21.15.17:0/2222301923 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f8d709de0 0x563f8d719840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.672 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.687+0000 7faf93fff700 1 -- 172.21.15.17:0/2222301923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563f8d7086d0 msgr2=0x563f8d7122c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T12:01:19.672 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.687+0000 7faf93fff700 1 --2- 172.21.15.17:0/2222301923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563f8d7086d0 0x563f8d7122c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.672 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.687+0000 7faf93fff700 1 -- 172.21.15.17:0/2222301923 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f8d709de0 msgr2=0x563f8d719840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T12:01:19.672 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.687+0000 7faf93fff700 1 --2- 172.21.15.17:0/2222301923 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f8d709de0 0x563f8d719840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.672 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.687+0000 7faf93fff700 1 -- 172.21.15.17:0/2222301923 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf9c19fdc0 con 0x563f8d709020 2023-07-14T12:01:19.673 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.687+0000 7faf9b7fe700 1 --2- 172.21.15.17:0/2222301923 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f8d709de0 0x563f8d719840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T12:01:19.673 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.687+0000 7faf93fff700 1 --2- 172.21.15.17:0/2222301923 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f8d709020 0x563f8d712800 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7faf84011790 tx=0x7faf84005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b70f2b11ad914bd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:19.673 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.687+0000 7faf827fc700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-07-14T12:01:19.673 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.687+0000 7faf827fc700 1 -- 172.21.15.17:0/2222301923 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf84004850 con 0x563f8d709020 2023-07-14T12:01:19.673 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.687+0000 7fafb4f21b80 1 -- 172.21.15.17:0/2222301923 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563f8d5b34a0 con 0x563f8d709020 2023-07-14T12:01:19.673 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.687+0000 7faf827fc700 1 -- 172.21.15.17:0/2222301923 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7faf84011c70 con 0x563f8d709020 2023-07-14T12:01:19.675 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.688+0000 7faf827fc700 1 -- 172.21.15.17:0/2222301923 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf84011e30 con 0x563f8d709020 2023-07-14T12:01:19.675 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.688+0000 7fafb4f21b80 10 client.15138 fetch_fsmap learned FSMap version 27 2023-07-14T12:01:19.675 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.688+0000 7fafb4f21b80 1 -- 172.21.15.17:0/2222301923 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7faf840098e0 con 0x563f8d709020 2023-07-14T12:01:19.675 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.688+0000 7faf827fc700 1 -- 172.21.15.17:0/2222301923 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563f8d5b34a0 con 0x563f8d709020 2023-07-14T12:01:19.675 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.688+0000 7faf827fc700 1 -- 172.21.15.17:0/2222301923 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7faf840039a0 con 0x563f8d709020 2023-07-14T12:01:19.676 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.688+0000 7fafb4f21b80 10 client.15138 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T12:01:19.676 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.688+0000 7fafb4f21b80 10 client.15138 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-14T12:01:19.676 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.688+0000 7fafb4f21b80 20 client.15138 populate_metadata read hostname 'smithi017' 2023-07-14T12:01:19.676 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.688+0000 7fafb4f21b80 1 --2- 172.21.15.17:0/2222301923 >> [v2:172.21.15.93:6838/1281342428,v1:172.21.15.93:6839/1281342428] conn(0x563f8d71d3d0 0x563f8d71c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.676 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.688+0000 7fafb4f21b80 4 client.15138 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T12:01:19.676 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.688+0000 7fafb4f21b80 1 -- 172.21.15.17:0/2222301923 --> [v2:172.21.15.93:6838/1281342428,v1:172.21.15.93:6839/1281342428] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563f8d652e60 con 0x563f8d71d3d0 2023-07-14T12:01:19.676 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.689+0000 7faf9affd700 1 --2- 172.21.15.17:0/2222301923 >> [v2:172.21.15.93:6838/1281342428,v1:172.21.15.93:6839/1281342428] conn(0x563f8d71d3d0 0x563f8d71c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.676 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.689+0000 7faf9affd700 1 --2- 172.21.15.17:0/2222301923 >> [v2:172.21.15.93:6838/1281342428,v1:172.21.15.93:6839/1281342428] conn(0x563f8d71d3d0 0x563f8d71c830 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7759100592f334c1 server_cookie=e538d2f85665a0f2 in_seq=0 out_seq=0 2023-07-14T12:01:19.677 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.689+0000 7faf827fc700 10 client.15138 ms_handle_connect on v2:172.21.15.93:6838/1281342428 2023-07-14T12:01:19.677 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.690+0000 7faf827fc700 1 -- 172.21.15.17:0/2222301923 <== mds.1 v2:172.21.15.93:6838/1281342428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563f8d652e60 con 0x563f8d71d3d0 2023-07-14T12:01:19.677 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.690+0000 7faf827fc700 10 client.15138 handle_command_reply: tid=0 2023-07-14T12:01:19.677 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.690+0000 7fafb4f21b80 1 client.15138 shutdown 2023-07-14T12:01:19.677 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.690+0000 7fafb4f21b80 1 -- 172.21.15.17:0/2222301923 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f8d709020 msgr2=0x563f8d712800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:19.677 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.690+0000 7fafb4f21b80 1 --2- 172.21.15.17:0/2222301923 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f8d709020 0x563f8d712800 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7faf84011790 tx=0x7faf84005ce0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.677 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.690+0000 7fafb4f21b80 1 -- 172.21.15.17:0/2222301923 shutdown_connections 2023-07-14T12:01:19.678 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.690+0000 7fafb4f21b80 1 --2- 172.21.15.17:0/2222301923 >> [v2:172.21.15.93:6838/1281342428,v1:172.21.15.93:6839/1281342428] conn(0x563f8d71d3d0 0x563f8d71c830 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.678 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.690+0000 7fafb4f21b80 1 --2- 172.21.15.17:0/2222301923 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f8d709de0 0x563f8d719840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.678 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.690+0000 7fafb4f21b80 1 --2- 172.21.15.17:0/2222301923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563f8d7086d0 0x563f8d7122c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.678 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.690+0000 7fafb4f21b80 1 --2- 172.21.15.17:0/2222301923 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f8d709020 0x563f8d712800 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.678 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.690+0000 7faf827fc700 0 client.15138 ms_handle_reset on v2:172.21.15.93:6838/1281342428 2023-07-14T12:01:19.678 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.690+0000 7fafb4f21b80 1 -- 172.21.15.17:0/2222301923 >> 172.21.15.17:0/2222301923 conn(0x563f8d6fdfa0 msgr2=0x563f8d702ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:19.678 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.690+0000 7fafb4f21b80 1 -- 172.21.15.17:0/2222301923 shutdown_connections 2023-07-14T12:01:19.678 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.690+0000 7fafb4f21b80 1 -- 172.21.15.17:0/2222301923 wait complete. 2023-07-14T12:01:19.679 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.690+0000 7fafb4f21b80 20 client.15138 trim_cache size 0 max 16384 2023-07-14T12:01:19.689 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.709+0000 7fafb4f21b80 1 Processor -- start 2023-07-14T12:01:19.689 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.709+0000 7fafb4f21b80 1 -- start start 2023-07-14T12:01:19.690 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.709+0000 7fafb4f21b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563f8d7086d0 0x563f8d710fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.690 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.709+0000 7fafb4f21b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f8d709020 0x563f8d719800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.690 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.709+0000 7fafb4f21b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f8d709de0 0x563f8d71c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.690 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.709+0000 7fafb4f21b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f8d4e9e20 con 0x563f8d709020 2023-07-14T12:01:19.690 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.709+0000 7fafb4f21b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f8d6184b0 con 0x563f8d709de0 2023-07-14T12:01:19.691 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.709+0000 7fafb4f21b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f8d71e6c0 con 0x563f8d7086d0 2023-07-14T12:01:19.691 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.710+0000 7faf93fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f8d709020 0x563f8d719800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.691 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.710+0000 7faf93fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f8d709020 0x563f8d719800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53798/0 (socket says 172.21.15.17:53798) 2023-07-14T12:01:19.691 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.710+0000 7faf93fff700 1 -- 172.21.15.17:0/68389366 learned_addr learned my addr 172.21.15.17:0/68389366 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:19.691 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.710+0000 7faf9b7fe700 1 --2- 172.21.15.17:0/68389366 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f8d709de0 0x563f8d71c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.691 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.710+0000 7faf9affd700 1 --2- 172.21.15.17:0/68389366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563f8d7086d0 0x563f8d710fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.691 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.710+0000 7faf93fff700 1 -- 172.21.15.17:0/68389366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563f8d7086d0 msgr2=0x563f8d710fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T12:01:19.692 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.710+0000 7faf93fff700 1 --2- 172.21.15.17:0/68389366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563f8d7086d0 0x563f8d710fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.692 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.710+0000 7faf93fff700 1 -- 172.21.15.17:0/68389366 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f8d709de0 msgr2=0x563f8d71c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T12:01:19.692 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.710+0000 7faf93fff700 1 --2- 172.21.15.17:0/68389366 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f8d709de0 0x563f8d71c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.692 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.710+0000 7faf93fff700 1 -- 172.21.15.17:0/68389366 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f8d470c10 con 0x563f8d709020 2023-07-14T12:01:19.692 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.710+0000 7faf9b7fe700 1 --2- 172.21.15.17:0/68389366 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f8d709de0 0x563f8d71c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T12:01:19.692 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.710+0000 7faf93fff700 1 --2- 172.21.15.17:0/68389366 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f8d709020 0x563f8d719800 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7faf84009290 tx=0x7faf84011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fb90d9a7d75b901 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:19.695 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.710+0000 7faf91ffb700 1 -- 172.21.15.17:0/68389366 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf84004a90 con 0x563f8d709020 2023-07-14T12:01:19.696 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.710+0000 7faf91ffb700 1 -- 172.21.15.17:0/68389366 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf84003b40 con 0x563f8d709020 2023-07-14T12:01:19.696 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.710+0000 7faf91ffb700 1 -- 172.21.15.17:0/68389366 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf84022c20 con 0x563f8d709020 2023-07-14T12:01:19.696 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.711+0000 7fafb4f21b80 1 -- 172.21.15.17:0/68389366 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f8d709020 msgr2=0x563f8d719800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:19.696 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.711+0000 7fafb4f21b80 1 --2- 172.21.15.17:0/68389366 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f8d709020 0x563f8d719800 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7faf84009290 tx=0x7faf84011440 comp rx=0 tx=0).stop 2023-07-14T12:01:19.696 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.711+0000 7fafb4f21b80 1 -- 172.21.15.17:0/68389366 shutdown_connections 2023-07-14T12:01:19.696 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.711+0000 7fafb4f21b80 1 --2- 172.21.15.17:0/68389366 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f8d709de0 0x563f8d71c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.696 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.711+0000 7fafb4f21b80 1 --2- 172.21.15.17:0/68389366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563f8d7086d0 0x563f8d710fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.697 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.711+0000 7fafb4f21b80 1 --2- 172.21.15.17:0/68389366 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f8d709020 0x563f8d719800 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.697 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.711+0000 7fafb4f21b80 1 -- 172.21.15.17:0/68389366 >> 172.21.15.17:0/68389366 conn(0x563f8d6fdfa0 msgr2=0x563f8d702ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:19.697 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.711+0000 7fafb4f21b80 1 -- 172.21.15.17:0/68389366 shutdown_connections 2023-07-14T12:01:19.697 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.711+0000 7fafb4f21b80 1 -- 172.21.15.17:0/68389366 wait complete. 2023-07-14T12:01:19.697 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.712+0000 7fafb4f21b80 1 -- start start 2023-07-14T12:01:19.701 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.712+0000 7fafb4f21b80 1 Processor -- start 2023-07-14T12:01:19.701 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.712+0000 7fafb4f21b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f8d7086d0 0x563f8d714a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.701 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.712+0000 7fafb4f21b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f8d709020 0x563f8d714fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.701 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.712+0000 7fafb4f21b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563f8d709de0 0x563f8d717ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.701 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.712+0000 7fafb4f21b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f8d718410 con 0x563f8d709020 2023-07-14T12:01:19.702 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.712+0000 7fafb4f21b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f8d718580 con 0x563f8d7086d0 2023-07-14T12:01:19.702 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.712+0000 7fafb4f21b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f8d718740 con 0x563f8d709de0 2023-07-14T12:01:19.702 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.712+0000 7fafb4f21b80 10 client.0 _ll_register_callbacks cb 0x7faf9400e4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T12:01:19.702 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.712+0000 7faf93fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f8d709020 0x563f8d714fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.702 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.713+0000 7faf93fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f8d709020 0x563f8d714fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53800/0 (socket says 172.21.15.17:53800) 2023-07-14T12:01:19.702 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.713+0000 7faf9affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f8d7086d0 0x563f8d714a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.702 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.713+0000 7faf93fff700 1 -- 172.21.15.17:0/853650786 learned_addr learned my addr 172.21.15.17:0/853650786 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:19.702 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.713+0000 7faf9b7fe700 1 --2- 172.21.15.17:0/853650786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563f8d709de0 0x563f8d717ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.703 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.713+0000 7faf9affd700 1 -- 172.21.15.17:0/853650786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563f8d709de0 msgr2=0x563f8d717ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T12:01:19.703 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.713+0000 7faf9affd700 1 --2- 172.21.15.17:0/853650786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563f8d709de0 0x563f8d717ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.703 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.713+0000 7faf9affd700 1 -- 172.21.15.17:0/853650786 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f8d709020 msgr2=0x563f8d714fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T12:01:19.703 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.713+0000 7faf9affd700 1 --2- 172.21.15.17:0/853650786 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f8d709020 0x563f8d714fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.703 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.713+0000 7faf9affd700 1 -- 172.21.15.17:0/853650786 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf840098e0 con 0x563f8d7086d0 2023-07-14T12:01:19.703 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.713+0000 7faf93fff700 1 --2- 172.21.15.17:0/853650786 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f8d709020 0x563f8d714fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T12:01:19.703 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.713+0000 7faf9affd700 1 --2- 172.21.15.17:0/853650786 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f8d7086d0 0x563f8d714a70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7faf8c09ae90 tx=0x7faf8c015b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a41b95588d682a9b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:19.704 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.713+0000 7faf827fc700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-14T12:01:19.704 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.713+0000 7faf827fc700 1 -- 172.21.15.17:0/853650786 <== mon.2 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c09f1c0 con 0x563f8d7086d0 2023-07-14T12:01:19.704 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.713+0000 7fafb4f21b80 1 -- 172.21.15.17:0/853650786 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faf84004510 con 0x563f8d7086d0 2023-07-14T12:01:19.704 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.713+0000 7faf827fc700 1 -- 172.21.15.17:0/853650786 <== mon.2 v2:172.21.15.93:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7faf8c0a3550 con 0x563f8d7086d0 2023-07-14T12:01:19.704 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.714+0000 7faf827fc700 1 -- 172.21.15.17:0/853650786 <== mon.2 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf8c0228e0 con 0x563f8d7086d0 2023-07-14T12:01:19.704 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.714+0000 7fafb4f21b80 10 client.25151 fetch_fsmap learned FSMap version 27 2023-07-14T12:01:19.704 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.714+0000 7fafb4f21b80 1 -- 172.21.15.17:0/853650786 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563f8d4ae200 con 0x563f8d7086d0 2023-07-14T12:01:19.705 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.714+0000 7faf827fc700 1 -- 172.21.15.17:0/853650786 <== mon.2 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf84004510 con 0x563f8d7086d0 2023-07-14T12:01:19.705 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.714+0000 7faf827fc700 1 -- 172.21.15.17:0/853650786 <== mon.2 v2:172.21.15.93:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7faf8c090be0 con 0x563f8d7086d0 2023-07-14T12:01:19.705 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.714+0000 7fafb4f21b80 10 client.25151 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T12:01:19.705 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.714+0000 7fafb4f21b80 10 client.25151 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-14T12:01:19.705 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.714+0000 7fafb4f21b80 20 client.25151 populate_metadata read hostname 'smithi017' 2023-07-14T12:01:19.705 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.714+0000 7fafb4f21b80 1 --2- 172.21.15.17:0/853650786 >> [v2:172.21.15.93:6838/1281342428,v1:172.21.15.93:6839/1281342428] conn(0x563f8d7765f0 0x563f8d7732a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.705 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.714+0000 7fafb4f21b80 4 client.25151 mds_command: new command op to 24557 tid=0[{"prefix": "status"}] 2023-07-14T12:01:19.706 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.714+0000 7fafb4f21b80 1 -- 172.21.15.17:0/853650786 --> [v2:172.21.15.93:6838/1281342428,v1:172.21.15.93:6839/1281342428] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563f8d615000 con 0x563f8d7765f0 2023-07-14T12:01:19.706 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.714+0000 7faf93fff700 1 --2- 172.21.15.17:0/853650786 >> [v2:172.21.15.93:6838/1281342428,v1:172.21.15.93:6839/1281342428] conn(0x563f8d7765f0 0x563f8d7732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.706 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.715+0000 7faf827fc700 10 client.25151 ms_handle_connect on v2:172.21.15.93:6838/1281342428 2023-07-14T12:01:19.706 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.715+0000 7faf93fff700 1 --2- 172.21.15.17:0/853650786 >> [v2:172.21.15.93:6838/1281342428,v1:172.21.15.93:6839/1281342428] conn(0x563f8d7765f0 0x563f8d7732a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=744ecb4170cb04fb server_cookie=b5735e6e7cb170f in_seq=0 out_seq=0 2023-07-14T12:01:19.706 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.715+0000 7faf827fc700 1 -- 172.21.15.17:0/853650786 <== mds.1 v2:172.21.15.93:6838/1281342428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x563f8d615000 con 0x563f8d7765f0 2023-07-14T12:01:19.706 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.715+0000 7faf827fc700 10 client.25151 handle_command_reply: tid=0 2023-07-14T12:01:19.706 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.715+0000 7fafb4f21b80 1 client.25151 shutdown 2023-07-14T12:01:19.706 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.715+0000 7fafb4f21b80 1 -- 172.21.15.17:0/853650786 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f8d7086d0 msgr2=0x563f8d714a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:19.707 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.715+0000 7fafb4f21b80 1 --2- 172.21.15.17:0/853650786 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f8d7086d0 0x563f8d714a70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7faf8c09ae90 tx=0x7faf8c015b70 comp rx=0 tx=0).stop 2023-07-14T12:01:19.707 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.715+0000 7fafb4f21b80 1 -- 172.21.15.17:0/853650786 shutdown_connections 2023-07-14T12:01:19.707 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.715+0000 7fafb4f21b80 1 --2- 172.21.15.17:0/853650786 >> [v2:172.21.15.93:6838/1281342428,v1:172.21.15.93:6839/1281342428] conn(0x563f8d7765f0 0x563f8d7732a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.707 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.715+0000 7fafb4f21b80 1 --2- 172.21.15.17:0/853650786 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f8d7086d0 0x563f8d714a70 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.707 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.715+0000 7fafb4f21b80 1 --2- 172.21.15.17:0/853650786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563f8d709de0 0x563f8d717ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.707 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.715+0000 7fafb4f21b80 1 --2- 172.21.15.17:0/853650786 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f8d709020 0x563f8d714fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.707 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.715+0000 7faf827fc700 0 client.25151 ms_handle_reset on v2:172.21.15.93:6838/1281342428 2023-07-14T12:01:19.707 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.715+0000 7fafb4f21b80 1 -- 172.21.15.17:0/853650786 >> 172.21.15.17:0/853650786 conn(0x563f8d6fdfa0 msgr2=0x563f8d710fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:19.708 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.715+0000 7fafb4f21b80 1 -- 172.21.15.17:0/853650786 shutdown_connections 2023-07-14T12:01:19.708 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.716+0000 7fafb4f21b80 1 -- 172.21.15.17:0/853650786 wait complete. 2023-07-14T12:01:19.708 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.716+0000 7fafb4f21b80 20 client.25151 trim_cache size 0 max 16384 2023-07-14T12:01:19.708 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2023-07-14T12:01:19.709 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "b3c386cc-223c-11ee-9b33-001a4aab830c", 2023-07-14T12:01:19.709 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 1, 2023-07-14T12:01:19.709 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 24557, 2023-07-14T12:01:19.709 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2023-07-14T12:01:19.709 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2023-07-14T12:01:19.709 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2023-07-14T12:01:19.709 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 66.262496349000003, 2023-07-14T12:01:19.710 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 27, 2023-07-14T12:01:19.710 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 92, 2023-07-14T12:01:19.710 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 90, 2023-07-14T12:01:19.710 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 102.18717388 2023-07-14T12:01:19.710 INFO:teuthology.orchestra.run.smithi017.stdout:} 2023-07-14T12:01:19.711 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.716+0000 7faf9bfff700 1 -- 172.21.15.17:0/2528732744 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7faf880a1150 msgr2=0x7faf880a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:19.711 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.716+0000 7faf9bfff700 1 --2- 172.21.15.17:0/2528732744 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7faf880a1150 0x7faf880a3600 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7faf9400ef00 tx=0x7faf94004d50 comp rx=0 tx=0).stop 2023-07-14T12:01:19.711 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.716+0000 7faf9bfff700 1 -- 172.21.15.17:0/2528732744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf9c068500 msgr2=0x7faf9c1978d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:19.711 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.716+0000 7faf9bfff700 1 --2- 172.21.15.17:0/2528732744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf9c068500 0x7faf9c1978d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7faf8c005720 tx=0x7faf8c009580 comp rx=0 tx=0).stop 2023-07-14T12:01:19.711 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.716+0000 7faf9bfff700 1 -- 172.21.15.17:0/2528732744 shutdown_connections 2023-07-14T12:01:19.711 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.716+0000 7faf9bfff700 1 --2- 172.21.15.17:0/2528732744 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7faf880a1150 0x7faf880a3600 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.712 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.716+0000 7faf9bfff700 1 --2- 172.21.15.17:0/2528732744 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf9c1054e0 0x7faf9c19ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.712 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.716+0000 7faf9bfff700 1 --2- 172.21.15.17:0/2528732744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf9c068500 0x7faf9c1978d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.712 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.716+0000 7faf9bfff700 1 --2- 172.21.15.17:0/2528732744 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faf9c1029f0 0x7faf9c197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.712 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.716+0000 7faf9bfff700 1 -- 172.21.15.17:0/2528732744 >> 172.21.15.17:0/2528732744 conn(0x7faf9c0f9de0 msgr2=0x7faf9c100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:19.712 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.716+0000 7faf9bfff700 1 -- 172.21.15.17:0/2528732744 shutdown_connections 2023-07-14T12:01:19.712 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.716+0000 7faf9bfff700 1 -- 172.21.15.17:0/2528732744 wait complete. 2023-07-14T12:01:19.714 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-07-14T12:01:19.921 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.941+0000 7efdda3ff700 1 -- 172.21.15.17:0/3704161622 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdd40ffbc0 msgr2=0x7efdd40fffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:19.921 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.941+0000 7efdda3ff700 1 --2- 172.21.15.17:0/3704161622 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdd40ffbc0 0x7efdd40fffd0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7efdc4009990 tx=0x7efdc4009ca0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.921 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.941+0000 7efdda3ff700 1 -- 172.21.15.17:0/3704161622 shutdown_connections 2023-07-14T12:01:19.922 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.941+0000 7efdda3ff700 1 --2- 172.21.15.17:0/3704161622 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efdd4101800 0x7efdd41080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.922 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.941+0000 7efdda3ff700 1 --2- 172.21.15.17:0/3704161622 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdd40ffbc0 0x7efdd40fffd0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.922 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.941+0000 7efdda3ff700 1 --2- 172.21.15.17:0/3704161622 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efdd4100e50 0x7efdd41012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.922 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.941+0000 7efdda3ff700 1 -- 172.21.15.17:0/3704161622 >> 172.21.15.17:0/3704161622 conn(0x7efdd40fb200 msgr2=0x7efdd40fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:19.922 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.941+0000 7efdda3ff700 1 -- 172.21.15.17:0/3704161622 shutdown_connections 2023-07-14T12:01:19.922 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.941+0000 7efdda3ff700 1 -- 172.21.15.17:0/3704161622 wait complete. 2023-07-14T12:01:19.924 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.942+0000 7efdda3ff700 1 Processor -- start 2023-07-14T12:01:19.924 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.942+0000 7efdda3ff700 1 -- start start 2023-07-14T12:01:19.924 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.942+0000 7efdda3ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdd40ffbc0 0x7efdd40742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.924 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.942+0000 7efdda3ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efdd4100e50 0x7efdd4072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.924 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.942+0000 7efdda3ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efdd4101800 0x7efdd4072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.925 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.942+0000 7efdda3ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd40735e0 con 0x7efdd40ffbc0 2023-07-14T12:01:19.925 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.942+0000 7efdda3ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd4073750 con 0x7efdd4100e50 2023-07-14T12:01:19.925 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.943+0000 7efdda3ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd419f920 con 0x7efdd4101800 2023-07-14T12:01:19.925 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.943+0000 7efdd3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdd40ffbc0 0x7efdd40742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.925 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.943+0000 7efdd37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efdd4100e50 0x7efdd4072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.925 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.943+0000 7efdd3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdd40ffbc0 0x7efdd40742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53816/0 (socket says 172.21.15.17:53816) 2023-07-14T12:01:19.925 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.943+0000 7efdd3fff700 1 -- 172.21.15.17:0/1164211749 learned_addr learned my addr 172.21.15.17:0/1164211749 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:19.925 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.943+0000 7efdd899c700 1 --2- 172.21.15.17:0/1164211749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efdd4101800 0x7efdd4072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.926 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.943+0000 7efdd3fff700 1 -- 172.21.15.17:0/1164211749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efdd4101800 msgr2=0x7efdd4072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:19.926 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.943+0000 7efdd3fff700 1 --2- 172.21.15.17:0/1164211749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efdd4101800 0x7efdd4072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.926 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.943+0000 7efdd3fff700 1 -- 172.21.15.17:0/1164211749 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efdd4100e50 msgr2=0x7efdd4072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:19.926 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.943+0000 7efdd3fff700 1 --2- 172.21.15.17:0/1164211749 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efdd4100e50 0x7efdd4072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.926 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.943+0000 7efdd3fff700 1 -- 172.21.15.17:0/1164211749 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdc40095f0 con 0x7efdd40ffbc0 2023-07-14T12:01:19.926 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.943+0000 7efdd3fff700 1 --2- 172.21.15.17:0/1164211749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdd40ffbc0 0x7efdd40742c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7efdc4004740 tx=0x7efdc4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:19.926 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.944+0000 7efdd17fa700 1 -- 172.21.15.17:0/1164211749 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc401d070 con 0x7efdd40ffbc0 2023-07-14T12:01:19.927 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.944+0000 7efdda3ff700 1 -- 172.21.15.17:0/1164211749 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdd419fba0 con 0x7efdd40ffbc0 2023-07-14T12:01:19.927 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.944+0000 7efdd17fa700 1 -- 172.21.15.17:0/1164211749 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdc4004830 con 0x7efdd40ffbc0 2023-07-14T12:01:19.927 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.944+0000 7efdd17fa700 1 -- 172.21.15.17:0/1164211749 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc4005450 con 0x7efdd40ffbc0 2023-07-14T12:01:19.927 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.944+0000 7efdda3ff700 1 -- 172.21.15.17:0/1164211749 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdd419ff60 con 0x7efdd40ffbc0 2023-07-14T12:01:19.927 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.947+0000 7efdd17fa700 1 -- 172.21.15.17:0/1164211749 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7efdc400fb00 con 0x7efdd40ffbc0 2023-07-14T12:01:19.928 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.948+0000 7efdedc29b80 1 Processor -- start 2023-07-14T12:01:19.928 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.948+0000 7efdd17fa700 1 --2- 172.21.15.17:0/1164211749 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7efdbc0a1150 0x7efdbc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.929 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.948+0000 7efdedc29b80 1 -- start start 2023-07-14T12:01:19.930 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.949+0000 7efdd17fa700 1 -- 172.21.15.17:0/1164211749 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7efdc40cde20 con 0x7efdd40ffbc0 2023-07-14T12:01:19.930 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.949+0000 7efdd899c700 1 --2- 172.21.15.17:0/1164211749 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7efdbc0a1150 0x7efdbc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.930 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.949+0000 7efdedc29b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c324c59480 0x55c324c59890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.930 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.949+0000 7efdedc29b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c324c59dd0 0x55c324c5a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.930 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.949+0000 7efdedc29b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c324c5ac20 0x55c324c5b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.931 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.949+0000 7efdedc29b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c324b5ec60 con 0x55c324c59480 2023-07-14T12:01:19.931 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.949+0000 7efdedc29b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c324bb19f0 con 0x55c324c5ac20 2023-07-14T12:01:19.931 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.949+0000 7efdedc29b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c324baae80 con 0x55c324c59dd0 2023-07-14T12:01:19.931 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.949+0000 7efdd3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c324c59480 0x55c324c59890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.931 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.949+0000 7efdd3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c324c59480 0x55c324c59890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53830/0 (socket says 172.21.15.17:53830) 2023-07-14T12:01:19.931 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.949+0000 7efdd3fff700 1 -- 172.21.15.17:0/1139217239 learned_addr learned my addr 172.21.15.17:0/1139217239 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:19.931 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.949+0000 7efdd37fe700 1 --2- 172.21.15.17:0/1139217239 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c324c59dd0 0x55c324c5a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.932 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.950+0000 7efdd899c700 1 --2- 172.21.15.17:0/1139217239 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c324c5ac20 0x55c324c5b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.932 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.950+0000 7efdd3fff700 1 -- 172.21.15.17:0/1139217239 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c324c59dd0 msgr2=0x55c324c5a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T12:01:19.932 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.950+0000 7efdd3fff700 1 --2- 172.21.15.17:0/1139217239 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c324c59dd0 0x55c324c5a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.932 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.950+0000 7efdd3fff700 1 -- 172.21.15.17:0/1139217239 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c324c5ac20 msgr2=0x55c324c5b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T12:01:19.932 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.950+0000 7efdd3fff700 1 --2- 172.21.15.17:0/1139217239 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c324c5ac20 0x55c324c5b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.932 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.950+0000 7efdd3fff700 1 -- 172.21.15.17:0/1139217239 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3249d8d90 con 0x55c324c59480 2023-07-14T12:01:19.932 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.950+0000 7efdd37fe700 1 --2- 172.21.15.17:0/1139217239 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c324c59dd0 0x55c324c5a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T12:01:19.932 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.950+0000 7efdd899c700 1 --2- 172.21.15.17:0/1164211749 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7efdbc0a1150 0x7efdbc0a3600 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efdd4074920 tx=0x7efdcc00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:19.933 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.950+0000 7efdd3fff700 1 --2- 172.21.15.17:0/1139217239 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c324c59480 0x55c324c59890 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efdc409af00 tx=0x7efdc409aff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b019a7200815515 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:19.933 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.950+0000 7efdba7fc700 1 -- 172.21.15.17:0/1139217239 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc4099310 con 0x55c324c59480 2023-07-14T12:01:19.933 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.950+0000 7efdba7fc700 1 -- 172.21.15.17:0/1139217239 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdc40055b0 con 0x55c324c59480 2023-07-14T12:01:19.933 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.950+0000 7efdba7fc700 1 -- 172.21.15.17:0/1139217239 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc4095b80 con 0x55c324c59480 2023-07-14T12:01:19.935 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.951+0000 7efdedc29b80 1 -- 172.21.15.17:0/1139217239 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c324c59480 msgr2=0x55c324c59890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:19.935 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.951+0000 7efdedc29b80 1 --2- 172.21.15.17:0/1139217239 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c324c59480 0x55c324c59890 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efdc409af00 tx=0x7efdc409aff0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.935 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.951+0000 7efdedc29b80 1 -- 172.21.15.17:0/1139217239 shutdown_connections 2023-07-14T12:01:19.936 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.951+0000 7efdedc29b80 1 --2- 172.21.15.17:0/1139217239 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c324c5ac20 0x55c324c5b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.936 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.951+0000 7efdedc29b80 1 --2- 172.21.15.17:0/1139217239 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c324c59480 0x55c324c59890 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.936 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.951+0000 7efdedc29b80 1 --2- 172.21.15.17:0/1139217239 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c324c59dd0 0x55c324c5a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.936 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.951+0000 7efdedc29b80 1 -- 172.21.15.17:0/1139217239 >> 172.21.15.17:0/1139217239 conn(0x55c324c4ed50 msgr2=0x55c324c54180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:19.936 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.951+0000 7efdedc29b80 1 -- 172.21.15.17:0/1139217239 shutdown_connections 2023-07-14T12:01:19.936 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.951+0000 7efdedc29b80 1 -- 172.21.15.17:0/1139217239 wait complete. 2023-07-14T12:01:19.936 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.952+0000 7efdedc29b80 1 -- start start 2023-07-14T12:01:19.937 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.952+0000 7efdedc29b80 1 Processor -- start 2023-07-14T12:01:19.937 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.952+0000 7efdedc29b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c324c59480 0x55c324c62d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.937 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.952+0000 7efdedc29b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c324c59dd0 0x55c324c632b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.937 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.952+0000 7efdedc29b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c324c5ac20 0x55c324c6a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.937 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.952+0000 7efdedc29b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c324b4aa70 con 0x55c324c59480 2023-07-14T12:01:19.937 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.952+0000 7efdedc29b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c324b4a770 con 0x55c324c59dd0 2023-07-14T12:01:19.937 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.952+0000 7efdedc29b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c324a2e8d0 con 0x55c324c5ac20 2023-07-14T12:01:19.938 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.952+0000 7efdedc29b80 10 client.0 _ll_register_callbacks cb 0x55c324b48710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T12:01:19.938 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.953+0000 7efdd3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c324c59480 0x55c324c62d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.938 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.953+0000 7efdd3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c324c59480 0x55c324c62d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53840/0 (socket says 172.21.15.17:53840) 2023-07-14T12:01:19.938 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.953+0000 7efdd3fff700 1 -- 172.21.15.17:0/315045654 learned_addr learned my addr 172.21.15.17:0/315045654 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:19.938 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.953+0000 7efdd899c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c324c5ac20 0x55c324c6a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.938 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.953+0000 7efdd37fe700 1 --2- 172.21.15.17:0/315045654 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c324c59dd0 0x55c324c632b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.938 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.953+0000 7efdd3fff700 1 -- 172.21.15.17:0/315045654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c324c5ac20 msgr2=0x55c324c6a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T12:01:19.938 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.953+0000 7efdd3fff700 1 --2- 172.21.15.17:0/315045654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c324c5ac20 0x55c324c6a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.939 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.953+0000 7efdd3fff700 1 -- 172.21.15.17:0/315045654 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c324c59dd0 msgr2=0x55c324c632b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T12:01:19.939 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.953+0000 7efdd3fff700 1 --2- 172.21.15.17:0/315045654 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c324c59dd0 0x55c324c632b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.939 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.953+0000 7efdd3fff700 1 -- 172.21.15.17:0/315045654 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdd419ff60 con 0x55c324c59480 2023-07-14T12:01:19.939 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.953+0000 7efdd3fff700 1 --2- 172.21.15.17:0/315045654 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c324c59480 0x55c324c62d70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7efdc40cb620 tx=0x7efdc4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df98640b9d42021 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:19.939 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.953+0000 7efdb37fe700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-07-14T12:01:19.939 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.953+0000 7efdb37fe700 1 -- 172.21.15.17:0/315045654 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc4099170 con 0x55c324c59480 2023-07-14T12:01:19.939 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.954+0000 7efdb37fe700 1 -- 172.21.15.17:0/315045654 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7efdc40aebe0 con 0x55c324c59480 2023-07-14T12:01:19.939 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.954+0000 7efdedc29b80 1 -- 172.21.15.17:0/315045654 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c324b044a0 con 0x55c324c59480 2023-07-14T12:01:19.940 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.954+0000 7efdb37fe700 1 -- 172.21.15.17:0/315045654 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7efdc40b9250 con 0x55c324c59480 2023-07-14T12:01:19.940 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.954+0000 7efdedc29b80 10 client.15156 fetch_fsmap learned FSMap version 27 2023-07-14T12:01:19.940 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.954+0000 7efdedc29b80 1 -- 172.21.15.17:0/315045654 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c3249c1c10 con 0x55c324c59480 2023-07-14T12:01:19.940 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.954+0000 7efdb37fe700 1 -- 172.21.15.17:0/315045654 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdc40cb020 con 0x55c324c59480 2023-07-14T12:01:19.940 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.954+0000 7efdb37fe700 1 -- 172.21.15.17:0/315045654 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efdc4095c10 con 0x55c324c59480 2023-07-14T12:01:19.940 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.954+0000 7efdedc29b80 10 client.15156 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T12:01:19.940 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.954+0000 7efdedc29b80 10 client.15156 resolve_mds: resolved name 'a' to daemon mds.a 2023-07-14T12:01:19.940 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.954+0000 7efdedc29b80 20 client.15156 populate_metadata read hostname 'smithi017' 2023-07-14T12:01:19.941 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.954+0000 7efdedc29b80 1 --2- 172.21.15.17:0/315045654 >> [v2:172.21.15.17:6840/3905201143,v1:172.21.15.17:6841/3905201143] conn(0x55c324c6e0c0 0x55c324c6d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.941 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.954+0000 7efdedc29b80 4 client.15156 mds_command: new command op to 24565 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T12:01:19.941 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.954+0000 7efdedc29b80 1 -- 172.21.15.17:0/315045654 --> [v2:172.21.15.17:6840/3905201143,v1:172.21.15.17:6841/3905201143] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c324ba3e60 con 0x55c324c6e0c0 2023-07-14T12:01:19.941 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.955+0000 7efdd37fe700 1 --2- 172.21.15.17:0/315045654 >> [v2:172.21.15.17:6840/3905201143,v1:172.21.15.17:6841/3905201143] conn(0x55c324c6e0c0 0x55c324c6d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.941 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.955+0000 7efdd37fe700 1 --2- 172.21.15.17:0/315045654 >> [v2:172.21.15.17:6840/3905201143,v1:172.21.15.17:6841/3905201143] conn(0x55c324c6e0c0 0x55c324c6d220 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e3601c85e8784b1 server_cookie=34475a299fc4056b in_seq=0 out_seq=0 2023-07-14T12:01:19.941 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.955+0000 7efdb37fe700 10 client.15156 ms_handle_connect on v2:172.21.15.17:6840/3905201143 2023-07-14T12:01:19.942 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.956+0000 7efdb37fe700 1 -- 172.21.15.17:0/315045654 <== mds.0 v2:172.21.15.17:6840/3905201143 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c324ba3e60 con 0x55c324c6e0c0 2023-07-14T12:01:19.942 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.956+0000 7efdb37fe700 10 client.15156 handle_command_reply: tid=0 2023-07-14T12:01:19.942 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.956+0000 7efdedc29b80 1 client.15156 shutdown 2023-07-14T12:01:19.943 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.956+0000 7efdedc29b80 1 -- 172.21.15.17:0/315045654 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c324c59480 msgr2=0x55c324c62d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:19.943 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.956+0000 7efdedc29b80 1 --2- 172.21.15.17:0/315045654 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c324c59480 0x55c324c62d70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7efdc40cb620 tx=0x7efdc4009310 comp rx=0 tx=0).stop 2023-07-14T12:01:19.943 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.956+0000 7efdedc29b80 1 -- 172.21.15.17:0/315045654 shutdown_connections 2023-07-14T12:01:19.943 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.956+0000 7efdedc29b80 1 --2- 172.21.15.17:0/315045654 >> [v2:172.21.15.17:6840/3905201143,v1:172.21.15.17:6841/3905201143] conn(0x55c324c6e0c0 0x55c324c6d220 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.943 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.956+0000 7efdedc29b80 1 --2- 172.21.15.17:0/315045654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c324c5ac20 0x55c324c6a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.943 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.956+0000 7efdedc29b80 1 --2- 172.21.15.17:0/315045654 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c324c59480 0x55c324c62d70 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.943 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.956+0000 7efdedc29b80 1 --2- 172.21.15.17:0/315045654 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c324c59dd0 0x55c324c632b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.944 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.956+0000 7efdb37fe700 0 client.15156 ms_handle_reset on v2:172.21.15.17:6840/3905201143 2023-07-14T12:01:19.944 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.956+0000 7efdedc29b80 1 -- 172.21.15.17:0/315045654 >> 172.21.15.17:0/315045654 conn(0x55c324c4ed50 msgr2=0x55c324c53860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:19.944 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.956+0000 7efdedc29b80 1 -- 172.21.15.17:0/315045654 shutdown_connections 2023-07-14T12:01:19.944 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.956+0000 7efdedc29b80 1 -- 172.21.15.17:0/315045654 wait complete. 2023-07-14T12:01:19.944 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.957+0000 7efdedc29b80 20 client.15156 trim_cache size 0 max 16384 2023-07-14T12:01:19.955 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.975+0000 7efdedc29b80 1 Processor -- start 2023-07-14T12:01:19.955 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.975+0000 7efdedc29b80 1 -- start start 2023-07-14T12:01:19.955 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.975+0000 7efdedc29b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c324c59480 0x55c324c76820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.955 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.975+0000 7efdedc29b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c324c59dd0 0x55c324c6a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.956 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.975+0000 7efdedc29b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c324c5ac20 0x55c324c6d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.956 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.975+0000 7efdedc29b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c324b54470 con 0x55c324c59dd0 2023-07-14T12:01:19.956 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.975+0000 7efdedc29b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c324b79790 con 0x55c324c59480 2023-07-14T12:01:19.956 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.975+0000 7efdedc29b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c324a3ade0 con 0x55c324c5ac20 2023-07-14T12:01:19.957 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.975+0000 7efdd37fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c324c59dd0 0x55c324c6a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.958 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.975+0000 7efdd37fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c324c59dd0 0x55c324c6a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53844/0 (socket says 172.21.15.17:53844) 2023-07-14T12:01:19.958 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.975+0000 7efdd37fe700 1 -- 172.21.15.17:0/3452246843 learned_addr learned my addr 172.21.15.17:0/3452246843 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:19.958 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.976+0000 7efdd3fff700 1 --2- 172.21.15.17:0/3452246843 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c324c59480 0x55c324c76820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.958 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.976+0000 7efdd899c700 1 --2- 172.21.15.17:0/3452246843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c324c5ac20 0x55c324c6d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.958 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.976+0000 7efdd37fe700 1 -- 172.21.15.17:0/3452246843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c324c5ac20 msgr2=0x55c324c6d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T12:01:19.958 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.976+0000 7efdd37fe700 1 --2- 172.21.15.17:0/3452246843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c324c5ac20 0x55c324c6d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.959 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.976+0000 7efdd37fe700 1 -- 172.21.15.17:0/3452246843 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c324c59480 msgr2=0x55c324c76820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T12:01:19.959 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.976+0000 7efdd37fe700 1 --2- 172.21.15.17:0/3452246843 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c324c59480 0x55c324c76820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.959 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.976+0000 7efdd37fe700 1 -- 172.21.15.17:0/3452246843 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3249ff200 con 0x55c324c59dd0 2023-07-14T12:01:19.959 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.976+0000 7efdd3fff700 1 --2- 172.21.15.17:0/3452246843 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c324c59480 0x55c324c76820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T12:01:19.959 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.976+0000 7efdd37fe700 1 --2- 172.21.15.17:0/3452246843 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c324c59dd0 0x55c324c6a280 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x55c324c6ea80 tx=0x7efdc8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d17b32709871dc4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:19.959 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.976+0000 7efdbaffd700 1 -- 172.21.15.17:0/3452246843 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc8017040 con 0x55c324c59dd0 2023-07-14T12:01:19.959 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.976+0000 7efdbaffd700 1 -- 172.21.15.17:0/3452246843 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdc8023070 con 0x55c324c59dd0 2023-07-14T12:01:19.959 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.976+0000 7efdbaffd700 1 -- 172.21.15.17:0/3452246843 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc8004e90 con 0x55c324c59dd0 2023-07-14T12:01:19.960 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.977+0000 7efdedc29b80 1 -- 172.21.15.17:0/3452246843 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c324c59dd0 msgr2=0x55c324c6a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:19.960 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.977+0000 7efdedc29b80 1 --2- 172.21.15.17:0/3452246843 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c324c59dd0 0x55c324c6a280 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x55c324c6ea80 tx=0x7efdc8009260 comp rx=0 tx=0).stop 2023-07-14T12:01:19.960 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.977+0000 7efdedc29b80 1 -- 172.21.15.17:0/3452246843 shutdown_connections 2023-07-14T12:01:19.960 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.977+0000 7efdedc29b80 1 --2- 172.21.15.17:0/3452246843 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c324c59480 0x55c324c76820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.960 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.977+0000 7efdedc29b80 1 --2- 172.21.15.17:0/3452246843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c324c5ac20 0x55c324c6d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.960 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.977+0000 7efdedc29b80 1 --2- 172.21.15.17:0/3452246843 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c324c59dd0 0x55c324c6a280 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.960 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.977+0000 7efdedc29b80 1 -- 172.21.15.17:0/3452246843 >> 172.21.15.17:0/3452246843 conn(0x55c324c4ed50 msgr2=0x55c324c502c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:19.960 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.977+0000 7efdedc29b80 1 -- 172.21.15.17:0/3452246843 shutdown_connections 2023-07-14T12:01:19.961 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.977+0000 7efdedc29b80 1 -- 172.21.15.17:0/3452246843 wait complete. 2023-07-14T12:01:19.961 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.977+0000 7efdedc29b80 1 -- start start 2023-07-14T12:01:19.961 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.978+0000 7efdedc29b80 1 Processor -- start 2023-07-14T12:01:19.963 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.978+0000 7efdedc29b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c324c59480 0x55c324c68270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.963 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.978+0000 7efdedc29b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c324c59dd0 0x55c324cbf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.963 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.978+0000 7efdedc29b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c324c5ac20 0x55c324cc1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.964 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.978+0000 7efdedc29b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c324cc23d0 con 0x55c324c5ac20 2023-07-14T12:01:19.964 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.978+0000 7efdedc29b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c324cc2540 con 0x55c324c59dd0 2023-07-14T12:01:19.964 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.978+0000 7efdedc29b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c324cc2700 con 0x55c324c59480 2023-07-14T12:01:19.964 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.978+0000 7efdedc29b80 10 client.0 _ll_register_callbacks cb 0x55c324b48710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T12:01:19.964 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.978+0000 7efdd3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c324c59480 0x55c324c68270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.964 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.978+0000 7efdd899c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c324c5ac20 0x55c324cc1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.964 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.978+0000 7efdd3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c324c59480 0x55c324c68270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.17:54468/0 (socket says 172.21.15.17:54468) 2023-07-14T12:01:19.965 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.978+0000 7efdd3fff700 1 -- 172.21.15.17:0/1123835699 learned_addr learned my addr 172.21.15.17:0/1123835699 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:19.965 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.979+0000 7efdd37fe700 1 --2- 172.21.15.17:0/1123835699 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c324c59dd0 0x55c324cbf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.965 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.979+0000 7efdd3fff700 1 -- 172.21.15.17:0/1123835699 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c324c59dd0 msgr2=0x55c324cbf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T12:01:19.965 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.979+0000 7efdd3fff700 1 --2- 172.21.15.17:0/1123835699 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c324c59dd0 0x55c324cbf940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.965 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.979+0000 7efdd3fff700 1 -- 172.21.15.17:0/1123835699 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c324c5ac20 msgr2=0x55c324cc1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T12:01:19.965 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.979+0000 7efdd3fff700 1 --2- 172.21.15.17:0/1123835699 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c324c5ac20 0x55c324cc1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.965 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.979+0000 7efdd3fff700 1 -- 172.21.15.17:0/1123835699 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdc8009a80 con 0x55c324c59480 2023-07-14T12:01:19.966 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.979+0000 7efdd899c700 1 --2- 172.21.15.17:0/1123835699 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c324c5ac20 0x55c324cc1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T12:01:19.966 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.979+0000 7efdd3fff700 1 --2- 172.21.15.17:0/1123835699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c324c59480 0x55c324c68270 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efdc409af00 tx=0x7efdc4009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5aed25302295f563 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:19.966 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.979+0000 7efdb37fe700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-07-14T12:01:19.966 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.979+0000 7efdb37fe700 1 -- 172.21.15.17:0/1123835699 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc4095d90 con 0x55c324c59480 2023-07-14T12:01:19.966 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.979+0000 7efdb37fe700 1 -- 172.21.15.17:0/1123835699 <== mon.1 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7efdc4005720 con 0x55c324c59480 2023-07-14T12:01:19.966 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.979+0000 7efdedc29b80 1 -- 172.21.15.17:0/1123835699 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efdc40055b0 con 0x55c324c59480 2023-07-14T12:01:19.966 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.980+0000 7efdb37fe700 1 -- 172.21.15.17:0/1123835699 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7efdc4099170 con 0x55c324c59480 2023-07-14T12:01:19.966 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.980+0000 7efdedc29b80 10 client.25186 fetch_fsmap learned FSMap version 27 2023-07-14T12:01:19.967 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.980+0000 7efdedc29b80 1 -- 172.21.15.17:0/1123835699 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c3249d7260 con 0x55c324c59480 2023-07-14T12:01:19.967 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.980+0000 7efdb37fe700 1 -- 172.21.15.17:0/1123835699 <== mon.1 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdc40055b0 con 0x55c324c59480 2023-07-14T12:01:19.967 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.980+0000 7efdb37fe700 1 -- 172.21.15.17:0/1123835699 <== mon.1 v2:172.21.15.97:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efdc409a460 con 0x55c324c59480 2023-07-14T12:01:19.967 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.980+0000 7efdedc29b80 10 client.25186 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T12:01:19.967 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.980+0000 7efdedc29b80 10 client.25186 resolve_mds: resolved name 'a' to daemon mds.a 2023-07-14T12:01:19.967 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.980+0000 7efdedc29b80 20 client.25186 populate_metadata read hostname 'smithi017' 2023-07-14T12:01:19.967 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.980+0000 7efdedc29b80 1 --2- 172.21.15.17:0/1123835699 >> [v2:172.21.15.17:6840/3905201143,v1:172.21.15.17:6841/3905201143] conn(0x55c324cc6ee0 0x55c324cc3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:19.967 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.980+0000 7efdedc29b80 4 client.25186 mds_command: new command op to 24565 tid=0[{"prefix": "status"}] 2023-07-14T12:01:19.968 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.980+0000 7efdedc29b80 1 -- 172.21.15.17:0/1123835699 --> [v2:172.21.15.17:6840/3905201143,v1:172.21.15.17:6841/3905201143] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c324b66000 con 0x55c324cc6ee0 2023-07-14T12:01:19.968 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.980+0000 7efdd37fe700 1 --2- 172.21.15.17:0/1123835699 >> [v2:172.21.15.17:6840/3905201143,v1:172.21.15.17:6841/3905201143] conn(0x55c324cc6ee0 0x55c324cc3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:19.968 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.981+0000 7efdd37fe700 1 --2- 172.21.15.17:0/1123835699 >> [v2:172.21.15.17:6840/3905201143,v1:172.21.15.17:6841/3905201143] conn(0x55c324cc6ee0 0x55c324cc3930 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=745f57df6d8884b8 server_cookie=ccf2043b96230e69 in_seq=0 out_seq=0 2023-07-14T12:01:19.968 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.981+0000 7efdb37fe700 10 client.25186 ms_handle_connect on v2:172.21.15.17:6840/3905201143 2023-07-14T12:01:19.968 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.981+0000 7efdb37fe700 1 -- 172.21.15.17:0/1123835699 <== mds.0 v2:172.21.15.17:6840/3905201143 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55c324b66000 con 0x55c324cc6ee0 2023-07-14T12:01:19.968 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.981+0000 7efdb37fe700 10 client.25186 handle_command_reply: tid=0 2023-07-14T12:01:19.968 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.981+0000 7efdedc29b80 1 client.25186 shutdown 2023-07-14T12:01:19.968 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.981+0000 7efdedc29b80 1 -- 172.21.15.17:0/1123835699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c324c59480 msgr2=0x55c324c68270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:19.969 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.981+0000 7efdedc29b80 1 --2- 172.21.15.17:0/1123835699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c324c59480 0x55c324c68270 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efdc409af00 tx=0x7efdc4009ca0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.969 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.981+0000 7efdedc29b80 1 -- 172.21.15.17:0/1123835699 shutdown_connections 2023-07-14T12:01:19.969 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.981+0000 7efdedc29b80 1 --2- 172.21.15.17:0/1123835699 >> [v2:172.21.15.17:6840/3905201143,v1:172.21.15.17:6841/3905201143] conn(0x55c324cc6ee0 0x55c324cc3930 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.969 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.981+0000 7efdedc29b80 1 --2- 172.21.15.17:0/1123835699 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c324c5ac20 0x55c324cc1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.969 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.981+0000 7efdedc29b80 1 --2- 172.21.15.17:0/1123835699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c324c59480 0x55c324c68270 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.969 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.981+0000 7efdb37fe700 0 client.25186 ms_handle_reset on v2:172.21.15.17:6840/3905201143 2023-07-14T12:01:19.969 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.981+0000 7efdedc29b80 1 --2- 172.21.15.17:0/1123835699 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c324c59dd0 0x55c324cbf940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.970 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.981+0000 7efdedc29b80 1 -- 172.21.15.17:0/1123835699 >> 172.21.15.17:0/1123835699 conn(0x55c324c4ed50 msgr2=0x55c324c76820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:19.970 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.982+0000 7efdedc29b80 1 -- 172.21.15.17:0/1123835699 shutdown_connections 2023-07-14T12:01:19.970 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.982+0000 7efdedc29b80 1 -- 172.21.15.17:0/1123835699 wait complete. 2023-07-14T12:01:19.970 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.982+0000 7efdedc29b80 20 client.25186 trim_cache size 0 max 16384 2023-07-14T12:01:19.970 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2023-07-14T12:01:19.970 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "b3c386cc-223c-11ee-9b33-001a4aab830c", 2023-07-14T12:01:19.971 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2023-07-14T12:01:19.971 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 24565, 2023-07-14T12:01:19.971 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2023-07-14T12:01:19.971 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2023-07-14T12:01:19.971 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2023-07-14T12:01:19.971 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 96.240335814999995, 2023-07-14T12:01:19.971 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 27, 2023-07-14T12:01:19.971 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 92, 2023-07-14T12:01:19.972 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 90, 2023-07-14T12:01:19.972 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 104.79539903200001 2023-07-14T12:01:19.972 INFO:teuthology.orchestra.run.smithi017.stdout:} 2023-07-14T12:01:19.972 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.982+0000 7efdda3ff700 1 -- 172.21.15.17:0/1164211749 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7efdbc0a1150 msgr2=0x7efdbc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:19.973 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.982+0000 7efdda3ff700 1 --2- 172.21.15.17:0/1164211749 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7efdbc0a1150 0x7efdbc0a3600 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efdd4074920 tx=0x7efdcc00b040 comp rx=0 tx=0).stop 2023-07-14T12:01:19.973 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.982+0000 7efdda3ff700 1 -- 172.21.15.17:0/1164211749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdd40ffbc0 msgr2=0x7efdd40742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:19.973 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.982+0000 7efdda3ff700 1 --2- 172.21.15.17:0/1164211749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdd40ffbc0 0x7efdd40742c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7efdc4004740 tx=0x7efdc4009580 comp rx=0 tx=0).stop 2023-07-14T12:01:19.973 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.982+0000 7efdda3ff700 1 -- 172.21.15.17:0/1164211749 shutdown_connections 2023-07-14T12:01:19.973 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.982+0000 7efdda3ff700 1 --2- 172.21.15.17:0/1164211749 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7efdbc0a1150 0x7efdbc0a3600 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.973 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.982+0000 7efdda3ff700 1 --2- 172.21.15.17:0/1164211749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efdd4101800 0x7efdd4072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.973 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.982+0000 7efdda3ff700 1 --2- 172.21.15.17:0/1164211749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdd40ffbc0 0x7efdd40742c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.974 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.982+0000 7efdda3ff700 1 --2- 172.21.15.17:0/1164211749 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efdd4100e50 0x7efdd4072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:19.974 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.982+0000 7efdda3ff700 1 -- 172.21.15.17:0/1164211749 >> 172.21.15.17:0/1164211749 conn(0x7efdd40fb200 msgr2=0x7efdd40fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:19.974 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.983+0000 7efdda3ff700 1 -- 172.21.15.17:0/1164211749 shutdown_connections 2023-07-14T12:01:19.974 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:19.983+0000 7efdda3ff700 1 -- 172.21.15.17:0/1164211749 wait complete. 2023-07-14T12:01:19.980 INFO:tasks.cephadm:Setup complete, yielding 2023-07-14T12:01:19.981 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-07-14T12:01:19.990 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi017.front.sepia.ceph.com 2023-07-14T12:01:19.990 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- bash -c 'ceph orch status' 2023-07-14T12:01:20.723 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T12:01:21.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:20 smithi093 ceph-mon[128922]: pgmap v329: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 9 B/s, 0 objects/s recovering 2023-07-14T12:01:21.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:20 smithi097 ceph-mon[129313]: pgmap v329: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 9 B/s, 0 objects/s recovering 2023-07-14T12:01:21.139 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:20 smithi017 ceph-mon[126339]: pgmap v329: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 9 B/s, 0 objects/s recovering 2023-07-14T12:01:22.248 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.267+0000 7fd8329e2700 1 -- 172.21.15.17:0/472013948 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd82c0ffb90 msgr2=0x7fd82c0fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:22.248 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.267+0000 7fd8329e2700 1 --2- 172.21.15.17:0/472013948 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd82c0ffb90 0x7fd82c0fffa0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd81c009990 tx=0x7fd81c009ca0 comp rx=0 tx=0).stop 2023-07-14T12:01:22.248 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.267+0000 7fd8329e2700 1 -- 172.21.15.17:0/472013948 shutdown_connections 2023-07-14T12:01:22.248 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.267+0000 7fd8329e2700 1 --2- 172.21.15.17:0/472013948 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd82c101720 0x7fd82c107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:22.248 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.267+0000 7fd8329e2700 1 --2- 172.21.15.17:0/472013948 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd82c0ffb90 0x7fd82c0fffa0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:22.249 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.267+0000 7fd8329e2700 1 --2- 172.21.15.17:0/472013948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd82c100d90 0x7fd82c1011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:22.249 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.267+0000 7fd8329e2700 1 -- 172.21.15.17:0/472013948 >> 172.21.15.17:0/472013948 conn(0x7fd82c0fb1a0 msgr2=0x7fd82c0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:22.249 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.267+0000 7fd8329e2700 1 -- 172.21.15.17:0/472013948 shutdown_connections 2023-07-14T12:01:22.249 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.267+0000 7fd8329e2700 1 -- 172.21.15.17:0/472013948 wait complete. 2023-07-14T12:01:22.251 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.268+0000 7fd8329e2700 1 Processor -- start 2023-07-14T12:01:22.251 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.269+0000 7fd8329e2700 1 -- start start 2023-07-14T12:01:22.251 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.269+0000 7fd8329e2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd82c0ffb90 0x7fd82c197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:22.252 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.269+0000 7fd8329e2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd82c100d90 0x7fd82c197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:22.252 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.269+0000 7fd8329e2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd82c101720 0x7fd82c19f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:22.252 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.269+0000 7fd8329e2700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82c19f630 con 0x7fd82c100d90 2023-07-14T12:01:22.252 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.269+0000 7fd8329e2700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82c19f770 con 0x7fd82c0ffb90 2023-07-14T12:01:22.253 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.269+0000 7fd8329e2700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82c19f900 con 0x7fd82c101720 2023-07-14T12:01:22.253 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.269+0000 7fd82b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd82c100d90 0x7fd82c197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:22.253 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.269+0000 7fd82b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd82c100d90 0x7fd82c197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53874/0 (socket says 172.21.15.17:53874) 2023-07-14T12:01:22.253 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.269+0000 7fd82b7fe700 1 -- 172.21.15.17:0/3753541606 learned_addr learned my addr 172.21.15.17:0/3753541606 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:22.254 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.269+0000 7fd82bfff700 1 --2- 172.21.15.17:0/3753541606 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd82c0ffb90 0x7fd82c197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:22.254 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.270+0000 7fd830f7f700 1 --2- 172.21.15.17:0/3753541606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd82c101720 0x7fd82c19f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:22.254 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.270+0000 7fd82b7fe700 1 -- 172.21.15.17:0/3753541606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd82c101720 msgr2=0x7fd82c19f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:22.254 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.270+0000 7fd82b7fe700 1 --2- 172.21.15.17:0/3753541606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd82c101720 0x7fd82c19f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:22.255 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.270+0000 7fd82b7fe700 1 -- 172.21.15.17:0/3753541606 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd82c0ffb90 msgr2=0x7fd82c197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:22.255 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.270+0000 7fd82b7fe700 1 --2- 172.21.15.17:0/3753541606 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd82c0ffb90 0x7fd82c197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:22.255 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.270+0000 7fd82b7fe700 1 -- 172.21.15.17:0/3753541606 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd81c0095f0 con 0x7fd82c100d90 2023-07-14T12:01:22.255 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.270+0000 7fd82bfff700 1 --2- 172.21.15.17:0/3753541606 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd82c0ffb90 0x7fd82c197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T12:01:22.256 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.270+0000 7fd82b7fe700 1 --2- 172.21.15.17:0/3753541606 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd82c100d90 0x7fd82c197fe0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd82000e9f0 tx=0x7fd82000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:22.256 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.270+0000 7fd8297fa700 1 -- 172.21.15.17:0/3753541606 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8200097b0 con 0x7fd82c100d90 2023-07-14T12:01:22.256 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.270+0000 7fd8297fa700 1 -- 172.21.15.17:0/3753541606 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd82000ee90 con 0x7fd82c100d90 2023-07-14T12:01:22.257 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.270+0000 7fd8329e2700 1 -- 172.21.15.17:0/3753541606 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd82c19fbe0 con 0x7fd82c100d90 2023-07-14T12:01:22.257 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.270+0000 7fd8297fa700 1 -- 172.21.15.17:0/3753541606 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd820005520 con 0x7fd82c100d90 2023-07-14T12:01:22.257 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.271+0000 7fd8329e2700 1 -- 172.21.15.17:0/3753541606 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd82c1a00e0 con 0x7fd82c100d90 2023-07-14T12:01:22.259 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.272+0000 7fd8329e2700 1 -- 172.21.15.17:0/3753541606 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd82c192230 con 0x7fd82c100d90 2023-07-14T12:01:22.259 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.274+0000 7fd8297fa700 1 -- 172.21.15.17:0/3753541606 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd820004750 con 0x7fd82c100d90 2023-07-14T12:01:22.259 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.275+0000 7fd8297fa700 1 --2- 172.21.15.17:0/3753541606 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fd8140a11d0 0x7fd8140a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:22.260 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.275+0000 7fd8297fa700 1 -- 172.21.15.17:0/3753541606 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fd820014070 con 0x7fd82c100d90 2023-07-14T12:01:22.260 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.275+0000 7fd82bfff700 1 --2- 172.21.15.17:0/3753541606 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fd8140a11d0 0x7fd8140a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:22.260 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.276+0000 7fd82bfff700 1 --2- 172.21.15.17:0/3753541606 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fd8140a11d0 0x7fd8140a3680 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd81c00b2e0 tx=0x7fd81c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:22.260 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.277+0000 7fd8297fa700 1 -- 172.21.15.17:0/3753541606 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd820096360 con 0x7fd82c100d90 2023-07-14T12:01:22.481 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.501+0000 7fd8329e2700 1 -- 172.21.15.17:0/3753541606 --> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd82c060950 con 0x7fd8140a11d0 2023-07-14T12:01:22.482 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.502+0000 7fd8297fa700 1 -- 172.21.15.17:0/3753541606 <== mgr.14152 v2:172.21.15.17:6800/4237449858 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fd82c060950 con 0x7fd8140a11d0 2023-07-14T12:01:22.482 INFO:teuthology.orchestra.run.smithi017.stdout:Backend: cephadm 2023-07-14T12:01:22.482 INFO:teuthology.orchestra.run.smithi017.stdout:Available: Yes 2023-07-14T12:01:22.482 INFO:teuthology.orchestra.run.smithi017.stdout:Paused: No 2023-07-14T12:01:22.484 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.504+0000 7fd8329e2700 1 -- 172.21.15.17:0/3753541606 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fd8140a11d0 msgr2=0x7fd8140a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:22.484 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.504+0000 7fd8329e2700 1 --2- 172.21.15.17:0/3753541606 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fd8140a11d0 0x7fd8140a3680 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd81c00b2e0 tx=0x7fd81c005fb0 comp rx=0 tx=0).stop 2023-07-14T12:01:22.485 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.504+0000 7fd8329e2700 1 -- 172.21.15.17:0/3753541606 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd82c100d90 msgr2=0x7fd82c197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:22.485 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.504+0000 7fd8329e2700 1 --2- 172.21.15.17:0/3753541606 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd82c100d90 0x7fd82c197fe0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd82000e9f0 tx=0x7fd82000ed00 comp rx=0 tx=0).stop 2023-07-14T12:01:22.485 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.504+0000 7fd8329e2700 1 -- 172.21.15.17:0/3753541606 shutdown_connections 2023-07-14T12:01:22.485 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.504+0000 7fd8329e2700 1 --2- 172.21.15.17:0/3753541606 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fd8140a11d0 0x7fd8140a3680 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:22.485 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.504+0000 7fd8329e2700 1 --2- 172.21.15.17:0/3753541606 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd82c0ffb90 0x7fd82c197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:22.485 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.504+0000 7fd8329e2700 1 --2- 172.21.15.17:0/3753541606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd82c101720 0x7fd82c19f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:22.485 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.504+0000 7fd8329e2700 1 --2- 172.21.15.17:0/3753541606 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd82c100d90 0x7fd82c197fe0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:22.486 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.504+0000 7fd8329e2700 1 -- 172.21.15.17:0/3753541606 >> 172.21.15.17:0/3753541606 conn(0x7fd82c0fb1a0 msgr2=0x7fd82c106710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:22.486 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.504+0000 7fd8329e2700 1 -- 172.21.15.17:0/3753541606 shutdown_connections 2023-07-14T12:01:22.486 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:22.504+0000 7fd8329e2700 1 -- 172.21.15.17:0/3753541606 wait complete. 2023-07-14T12:01:23.026 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- bash -c 'ceph orch ps' 2023-07-14T12:01:23.091 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:22 smithi017 ceph-mon[126339]: pgmap v330: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 9 B/s, 0 objects/s recovering 2023-07-14T12:01:23.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:22 smithi093 ceph-mon[128922]: pgmap v330: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 9 B/s, 0 objects/s recovering 2023-07-14T12:01:23.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:22 smithi097 ceph-mon[129313]: pgmap v330: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 9 B/s, 0 objects/s recovering 2023-07-14T12:01:23.760 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T12:01:24.086 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:23 smithi017 ceph-mon[126339]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T12:01:24.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:23 smithi093 ceph-mon[128922]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T12:01:24.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:23 smithi097 ceph-mon[129313]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T12:01:25.072 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:24 smithi017 ceph-mon[126339]: pgmap v331: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-07-14T12:01:25.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:24 smithi093 ceph-mon[128922]: pgmap v331: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-07-14T12:01:25.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:24 smithi097 ceph-mon[129313]: pgmap v331: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-07-14T12:01:25.330 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.350+0000 7fbc5c37f700 1 -- 172.21.15.17:0/3144081144 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc54100d90 msgr2=0x7fbc541011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:25.330 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.350+0000 7fbc5c37f700 1 --2- 172.21.15.17:0/3144081144 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc54100d90 0x7fbc541011e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbc44009990 tx=0x7fbc44009ca0 comp rx=0 tx=0).stop 2023-07-14T12:01:25.331 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.350+0000 7fbc5c37f700 1 -- 172.21.15.17:0/3144081144 shutdown_connections 2023-07-14T12:01:25.331 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.350+0000 7fbc5c37f700 1 --2- 172.21.15.17:0/3144081144 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbc54101720 0x7fbc54107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:25.331 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.350+0000 7fbc5c37f700 1 --2- 172.21.15.17:0/3144081144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc540ffb90 0x7fbc540fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:25.331 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.350+0000 7fbc5c37f700 1 --2- 172.21.15.17:0/3144081144 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc54100d90 0x7fbc541011e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:25.331 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.350+0000 7fbc5c37f700 1 -- 172.21.15.17:0/3144081144 >> 172.21.15.17:0/3144081144 conn(0x7fbc540fb1a0 msgr2=0x7fbc540fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:25.331 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.350+0000 7fbc5c37f700 1 -- 172.21.15.17:0/3144081144 shutdown_connections 2023-07-14T12:01:25.331 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.350+0000 7fbc5c37f700 1 -- 172.21.15.17:0/3144081144 wait complete. 2023-07-14T12:01:25.333 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.351+0000 7fbc5c37f700 1 Processor -- start 2023-07-14T12:01:25.333 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.352+0000 7fbc5c37f700 1 -- start start 2023-07-14T12:01:25.333 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.352+0000 7fbc5c37f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbc540ffb90 0x7fbc541958b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:25.333 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.352+0000 7fbc5c37f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc54100d90 0x7fbc54195df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:25.333 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.352+0000 7fbc5c37f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc54101720 0x7fbc5419ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:25.333 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.352+0000 7fbc5c37f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5419d440 con 0x7fbc54101720 2023-07-14T12:01:25.334 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.352+0000 7fbc5c37f700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5419d580 con 0x7fbc540ffb90 2023-07-14T12:01:25.334 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.352+0000 7fbc5c37f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5419d710 con 0x7fbc54100d90 2023-07-14T12:01:25.334 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.352+0000 7fbc5a11b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbc540ffb90 0x7fbc541958b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:25.334 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.352+0000 7fbc5991a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc54100d90 0x7fbc54195df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:25.334 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.352+0000 7fbc5a11b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbc540ffb90 0x7fbc541958b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.17:57066/0 (socket says 172.21.15.17:57066) 2023-07-14T12:01:25.334 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.352+0000 7fbc5a11b700 1 -- 172.21.15.17:0/3859086896 learned_addr learned my addr 172.21.15.17:0/3859086896 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:25.334 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.352+0000 7fbc5a91c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc54101720 0x7fbc5419ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:25.335 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.353+0000 7fbc5a11b700 1 -- 172.21.15.17:0/3859086896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc54100d90 msgr2=0x7fbc54195df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:25.335 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.353+0000 7fbc5a11b700 1 --2- 172.21.15.17:0/3859086896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc54100d90 0x7fbc54195df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:25.335 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.353+0000 7fbc5a11b700 1 -- 172.21.15.17:0/3859086896 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc54101720 msgr2=0x7fbc5419ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:25.335 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.353+0000 7fbc5a11b700 1 --2- 172.21.15.17:0/3859086896 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc54101720 0x7fbc5419ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:25.335 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.353+0000 7fbc5a11b700 1 -- 172.21.15.17:0/3859086896 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc440095f0 con 0x7fbc540ffb90 2023-07-14T12:01:25.335 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.353+0000 7fbc5991a700 1 --2- 172.21.15.17:0/3859086896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc54100d90 0x7fbc54195df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T12:01:25.335 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.353+0000 7fbc5a91c700 1 --2- 172.21.15.17:0/3859086896 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc54101720 0x7fbc5419ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T12:01:25.336 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.353+0000 7fbc5a11b700 1 --2- 172.21.15.17:0/3859086896 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbc540ffb90 0x7fbc541958b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbc5000e970 tx=0x7fbc5000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:25.336 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.353+0000 7fbc4b7fe700 1 -- 172.21.15.17:0/3859086896 <== mon.2 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc50009790 con 0x7fbc540ffb90 2023-07-14T12:01:25.337 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.354+0000 7fbc5c37f700 1 -- 172.21.15.17:0/3859086896 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc5419d9f0 con 0x7fbc540ffb90 2023-07-14T12:01:25.337 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.354+0000 7fbc4b7fe700 1 -- 172.21.15.17:0/3859086896 <== mon.2 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc5000c7e0 con 0x7fbc540ffb90 2023-07-14T12:01:25.337 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.354+0000 7fbc4b7fe700 1 -- 172.21.15.17:0/3859086896 <== mon.2 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc50003c70 con 0x7fbc540ffb90 2023-07-14T12:01:25.337 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.354+0000 7fbc5c37f700 1 -- 172.21.15.17:0/3859086896 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc5419df40 con 0x7fbc540ffb90 2023-07-14T12:01:25.337 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.355+0000 7fbc5c37f700 1 -- 172.21.15.17:0/3859086896 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc54107130 con 0x7fbc540ffb90 2023-07-14T12:01:25.338 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.357+0000 7fbc4b7fe700 1 -- 172.21.15.17:0/3859086896 <== mon.2 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fbc5000c950 con 0x7fbc540ffb90 2023-07-14T12:01:25.338 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.358+0000 7fbc4b7fe700 1 --2- 172.21.15.17:0/3859086896 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fbc400a1150 0x7fbc400a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:25.338 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.358+0000 7fbc4b7fe700 1 -- 172.21.15.17:0/3859086896 <== mon.2 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fbc50014070 con 0x7fbc540ffb90 2023-07-14T12:01:25.339 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.358+0000 7fbc5991a700 1 --2- 172.21.15.17:0/3859086896 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fbc400a1150 0x7fbc400a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:25.339 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.359+0000 7fbc5991a700 1 --2- 172.21.15.17:0/3859086896 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fbc400a1150 0x7fbc400a3600 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbc44006010 tx=0x7fbc44005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:25.340 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.360+0000 7fbc4b7fe700 1 -- 172.21.15.17:0/3859086896 <== mon.2 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fbc50096870 con 0x7fbc540ffb90 2023-07-14T12:01:25.559 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.579+0000 7fbc5c37f700 1 -- 172.21.15.17:0/3859086896 --> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc54060950 con 0x7fbc400a1150 2023-07-14T12:01:25.572 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.592+0000 7fbc4b7fe700 1 -- 172.21.15.17:0/3859086896 <== mgr.14152 v2:172.21.15.17:6800/4237449858 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7fbc54060950 con 0x7fbc400a1150 2023-07-14T12:01:25.573 INFO:teuthology.orchestra.run.smithi017.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-07-14T12:01:25.573 INFO:teuthology.orchestra.run.smithi017.stdout:mds.a smithi017 running (110s) 25s ago 110s 21.7M - 18.0.0-4757-g3fc0089c afd12e232ec0 815e6d261378 2023-07-14T12:01:25.573 INFO:teuthology.orchestra.run.smithi017.stdout:mds.b smithi093 running (108s) 37s ago 108s 18.3M - 18.0.0-4757-g3fc0089c afd12e232ec0 f90a320fdfd5 2023-07-14T12:01:25.573 INFO:teuthology.orchestra.run.smithi017.stdout:mds.c smithi097 running (2m) 37s ago 2m 20.2M - 18.0.0-4757-g3fc0089c afd12e232ec0 e18102a30fe4 2023-07-14T12:01:25.573 INFO:teuthology.orchestra.run.smithi017.stdout:mds.d smithi017 running (2m) 25s ago 2m 13.5M - 18.0.0-4757-g3fc0089c afd12e232ec0 2f4e1cd1dc59 2023-07-14T12:01:25.574 INFO:teuthology.orchestra.run.smithi017.stdout:mds.e smithi093 running (2m) 37s ago 2m 18.2M - 18.0.0-4757-g3fc0089c afd12e232ec0 0638049837c0 2023-07-14T12:01:25.574 INFO:teuthology.orchestra.run.smithi017.stdout:mds.f smithi097 running (2m) 37s ago 2m 16.9M - 18.0.0-4757-g3fc0089c afd12e232ec0 6df1162247ef 2023-07-14T12:01:25.574 INFO:teuthology.orchestra.run.smithi017.stdout:mds.g smithi017 running (114s) 25s ago 113s 16.8M - 18.0.0-4757-g3fc0089c afd12e232ec0 77724550bd9b 2023-07-14T12:01:25.574 INFO:teuthology.orchestra.run.smithi017.stdout:mds.h smithi093 running (119s) 37s ago 118s 19.2M - 18.0.0-4757-g3fc0089c afd12e232ec0 a4010356a2bf 2023-07-14T12:01:25.574 INFO:teuthology.orchestra.run.smithi017.stdout:mds.i smithi097 running (117s) 37s ago 116s 17.9M - 18.0.0-4757-g3fc0089c afd12e232ec0 31e2f76bdba4 2023-07-14T12:01:25.575 INFO:teuthology.orchestra.run.smithi017.stdout:mds.j smithi017 running (2m) 25s ago 2m 13.5M - 18.0.0-4757-g3fc0089c afd12e232ec0 be4008277c6a 2023-07-14T12:01:25.575 INFO:teuthology.orchestra.run.smithi017.stdout:mds.k smithi093 running (2m) 37s ago 2m 16.9M - 18.0.0-4757-g3fc0089c afd12e232ec0 6e44de260e66 2023-07-14T12:01:25.575 INFO:teuthology.orchestra.run.smithi017.stdout:mds.l smithi097 running (2m) 37s ago 2m 16.9M - 18.0.0-4757-g3fc0089c afd12e232ec0 ec425c55ba1c 2023-07-14T12:01:25.575 INFO:teuthology.orchestra.run.smithi017.stdout:mgr.x smithi017 *:9283,8765 running (9m) 25s ago 9m 521M - 18.0.0-4757-g3fc0089c afd12e232ec0 37fdbd61b165 2023-07-14T12:01:25.575 INFO:teuthology.orchestra.run.smithi017.stdout:mgr.y smithi093 *:8443,8765 running (7m) 37s ago 7m 448M - 18.0.0-4757-g3fc0089c afd12e232ec0 4d7dc8d530ca 2023-07-14T12:01:25.575 INFO:teuthology.orchestra.run.smithi017.stdout:mgr.z smithi097 *:8443,8765 running (7m) 37s ago 7m 446M - 18.0.0-4757-g3fc0089c afd12e232ec0 67ac8386d724 2023-07-14T12:01:25.575 INFO:teuthology.orchestra.run.smithi017.stdout:mon.a smithi017 running (9m) 25s ago 9m 92.7M 2048M 18.0.0-4757-g3fc0089c afd12e232ec0 a436742b0c29 2023-07-14T12:01:25.576 INFO:teuthology.orchestra.run.smithi017.stdout:mon.b smithi093 running (7m) 37s ago 7m 62.3M 2048M 18.0.0-4757-g3fc0089c afd12e232ec0 1030cf3d22ca 2023-07-14T12:01:25.576 INFO:teuthology.orchestra.run.smithi017.stdout:mon.c smithi097 running (7m) 37s ago 7m 66.2M 2048M 18.0.0-4757-g3fc0089c afd12e232ec0 afacccf97ff5 2023-07-14T12:01:25.576 INFO:teuthology.orchestra.run.smithi017.stdout:osd.0 smithi017 running (6m) 25s ago 6m 74.3M 4318M 18.0.0-4757-g3fc0089c afd12e232ec0 bce8d4d07fb0 2023-07-14T12:01:25.576 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1 smithi093 running (6m) 37s ago 6m 76.3M 4292M 18.0.0-4757-g3fc0089c afd12e232ec0 86547c9380ec 2023-07-14T12:01:25.576 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2 smithi097 running (6m) 37s ago 6m 73.8M 4292M 18.0.0-4757-g3fc0089c afd12e232ec0 6ee1b48bb710 2023-07-14T12:01:25.576 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3 smithi017 running (5m) 25s ago 5m 66.7M 4318M 18.0.0-4757-g3fc0089c afd12e232ec0 3c88335f6378 2023-07-14T12:01:25.576 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4 smithi093 running (5m) 37s ago 5m 65.2M 4292M 18.0.0-4757-g3fc0089c afd12e232ec0 bd99eb7805d5 2023-07-14T12:01:25.577 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5 smithi097 running (4m) 37s ago 4m 71.6M 4292M 18.0.0-4757-g3fc0089c afd12e232ec0 777d40b34055 2023-07-14T12:01:25.577 INFO:teuthology.orchestra.run.smithi017.stdout:osd.6 smithi017 running (4m) 25s ago 4m 64.0M 4318M 18.0.0-4757-g3fc0089c afd12e232ec0 c5210c9f83b9 2023-07-14T12:01:25.577 INFO:teuthology.orchestra.run.smithi017.stdout:osd.7 smithi093 running (4m) 37s ago 4m 65.7M 4292M 18.0.0-4757-g3fc0089c afd12e232ec0 675dbab7aba3 2023-07-14T12:01:25.577 INFO:teuthology.orchestra.run.smithi017.stdout:osd.8 smithi097 running (3m) 37s ago 3m 63.6M 4292M 18.0.0-4757-g3fc0089c afd12e232ec0 3fe57e237401 2023-07-14T12:01:25.577 INFO:teuthology.orchestra.run.smithi017.stdout:osd.9 smithi017 running (3m) 25s ago 3m 59.4M 4318M 18.0.0-4757-g3fc0089c afd12e232ec0 3ecb7a74418d 2023-07-14T12:01:25.577 INFO:teuthology.orchestra.run.smithi017.stdout:osd.10 smithi093 running (2m) 37s ago 2m 60.9M 4292M 18.0.0-4757-g3fc0089c afd12e232ec0 f348dd08cd23 2023-07-14T12:01:25.577 INFO:teuthology.orchestra.run.smithi017.stdout:osd.11 smithi097 running (2m) 37s ago 2m 56.0M 4292M 18.0.0-4757-g3fc0089c afd12e232ec0 8b55032a8801 2023-07-14T12:01:25.578 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.595+0000 7fbc5c37f700 1 -- 172.21.15.17:0/3859086896 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fbc400a1150 msgr2=0x7fbc400a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:25.578 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.595+0000 7fbc5c37f700 1 --2- 172.21.15.17:0/3859086896 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fbc400a1150 0x7fbc400a3600 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbc44006010 tx=0x7fbc44005a90 comp rx=0 tx=0).stop 2023-07-14T12:01:25.579 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.595+0000 7fbc5c37f700 1 -- 172.21.15.17:0/3859086896 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbc540ffb90 msgr2=0x7fbc541958b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:25.579 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.595+0000 7fbc5c37f700 1 --2- 172.21.15.17:0/3859086896 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbc540ffb90 0x7fbc541958b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbc5000e970 tx=0x7fbc5000ec80 comp rx=0 tx=0).stop 2023-07-14T12:01:25.579 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.595+0000 7fbc5c37f700 1 -- 172.21.15.17:0/3859086896 shutdown_connections 2023-07-14T12:01:25.579 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.595+0000 7fbc5c37f700 1 --2- 172.21.15.17:0/3859086896 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fbc400a1150 0x7fbc400a3600 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:25.579 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.595+0000 7fbc5c37f700 1 --2- 172.21.15.17:0/3859086896 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbc540ffb90 0x7fbc541958b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:25.579 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.595+0000 7fbc5c37f700 1 --2- 172.21.15.17:0/3859086896 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc54101720 0x7fbc5419ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:25.580 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.595+0000 7fbc5c37f700 1 --2- 172.21.15.17:0/3859086896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc54100d90 0x7fbc54195df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:25.580 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.595+0000 7fbc5c37f700 1 -- 172.21.15.17:0/3859086896 >> 172.21.15.17:0/3859086896 conn(0x7fbc540fb1a0 msgr2=0x7fbc54104270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:25.580 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.595+0000 7fbc5c37f700 1 -- 172.21.15.17:0/3859086896 shutdown_connections 2023-07-14T12:01:25.580 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:25.595+0000 7fbc5c37f700 1 -- 172.21.15.17:0/3859086896 wait complete. 2023-07-14T12:01:26.056 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- bash -c 'ceph orch ls' 2023-07-14T12:01:26.794 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T12:01:27.097 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:26 smithi017 ceph-mon[126339]: from='client.25181 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T12:01:27.098 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:26 smithi017 ceph-mon[126339]: pgmap v332: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-14T12:01:27.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:26 smithi093 ceph-mon[128922]: from='client.25181 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T12:01:27.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:26 smithi093 ceph-mon[128922]: pgmap v332: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-14T12:01:27.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:26 smithi097 ceph-mon[129313]: from='client.25181 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T12:01:27.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:26 smithi097 ceph-mon[129313]: pgmap v332: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-14T12:01:28.295 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.315+0000 7f840593a700 1 -- 172.21.15.17:0/211731181 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84000ffb90 msgr2=0x7f84000fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:28.296 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.315+0000 7f840593a700 1 --2- 172.21.15.17:0/211731181 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84000ffb90 0x7f84000fffa0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f83f0009990 tx=0x7f83f0009ca0 comp rx=0 tx=0).stop 2023-07-14T12:01:28.296 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.315+0000 7f840593a700 1 -- 172.21.15.17:0/211731181 shutdown_connections 2023-07-14T12:01:28.296 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.315+0000 7f840593a700 1 --2- 172.21.15.17:0/211731181 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8400101720 0x7f8400107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:28.297 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.315+0000 7f840593a700 1 --2- 172.21.15.17:0/211731181 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84000ffb90 0x7f84000fffa0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:28.297 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.315+0000 7f840593a700 1 --2- 172.21.15.17:0/211731181 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8400100d90 0x7f84001011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:28.297 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.315+0000 7f840593a700 1 -- 172.21.15.17:0/211731181 >> 172.21.15.17:0/211731181 conn(0x7f84000fb1a0 msgr2=0x7f84000fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:28.298 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.315+0000 7f840593a700 1 -- 172.21.15.17:0/211731181 shutdown_connections 2023-07-14T12:01:28.298 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.315+0000 7f840593a700 1 -- 172.21.15.17:0/211731181 wait complete. 2023-07-14T12:01:28.299 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.316+0000 7f840593a700 1 Processor -- start 2023-07-14T12:01:28.299 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.316+0000 7f840593a700 1 -- start start 2023-07-14T12:01:28.300 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.316+0000 7f840593a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84000ffb90 0x7f8400197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:28.300 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.316+0000 7f840593a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8400100d90 0x7f8400197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:28.300 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.316+0000 7f840593a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8400101720 0x7f840019f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:28.300 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.316+0000 7f840593a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840019f640 con 0x7f8400100d90 2023-07-14T12:01:28.300 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.316+0000 7f840593a700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840019f780 con 0x7f84000ffb90 2023-07-14T12:01:28.300 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.316+0000 7f840593a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840019f910 con 0x7f8400101720 2023-07-14T12:01:28.300 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.317+0000 7f83fe7fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8400100d90 0x7f8400197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:28.301 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.317+0000 7f83feffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84000ffb90 0x7f8400197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:28.301 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.317+0000 7f83fe7fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8400100d90 0x7f8400197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47984/0 (socket says 172.21.15.17:47984) 2023-07-14T12:01:28.301 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.317+0000 7f83fe7fc700 1 -- 172.21.15.17:0/3660617744 learned_addr learned my addr 172.21.15.17:0/3660617744 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:28.301 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.317+0000 7f83ff7fe700 1 --2- 172.21.15.17:0/3660617744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8400101720 0x7f840019f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:28.301 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.317+0000 7f83feffd700 1 -- 172.21.15.17:0/3660617744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8400101720 msgr2=0x7f840019f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:28.301 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.317+0000 7f83feffd700 1 --2- 172.21.15.17:0/3660617744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8400101720 0x7f840019f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:28.301 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.317+0000 7f83feffd700 1 -- 172.21.15.17:0/3660617744 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8400100d90 msgr2=0x7f8400197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:28.301 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.317+0000 7f83feffd700 1 --2- 172.21.15.17:0/3660617744 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8400100d90 0x7f8400197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:28.302 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.317+0000 7f83feffd700 1 -- 172.21.15.17:0/3660617744 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f00095f0 con 0x7f84000ffb90 2023-07-14T12:01:28.302 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.318+0000 7f83feffd700 1 --2- 172.21.15.17:0/3660617744 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84000ffb90 0x7f8400197ab0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f83f00098e0 tx=0x7f83f0009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:28.302 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.318+0000 7f8404938700 1 -- 172.21.15.17:0/3660617744 <== mon.2 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f001d070 con 0x7f84000ffb90 2023-07-14T12:01:28.302 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.318+0000 7f8404938700 1 -- 172.21.15.17:0/3660617744 <== mon.2 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83f0021410 con 0x7f84000ffb90 2023-07-14T12:01:28.302 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.318+0000 7f840593a700 1 -- 172.21.15.17:0/3660617744 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f840019fb90 con 0x7f84000ffb90 2023-07-14T12:01:28.303 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.318+0000 7f8404938700 1 -- 172.21.15.17:0/3660617744 <== mon.2 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f0017650 con 0x7f84000ffb90 2023-07-14T12:01:28.303 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.318+0000 7f840593a700 1 -- 172.21.15.17:0/3660617744 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f840019ffa0 con 0x7f84000ffb90 2023-07-14T12:01:28.303 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.319+0000 7f840593a700 1 -- 172.21.15.17:0/3660617744 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8400192210 con 0x7f84000ffb90 2023-07-14T12:01:28.304 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.321+0000 7f8404938700 1 -- 172.21.15.17:0/3660617744 <== mon.2 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f83f0021810 con 0x7f84000ffb90 2023-07-14T12:01:28.304 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.323+0000 7f8404938700 1 --2- 172.21.15.17:0/3660617744 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f83e80a1150 0x7f83e80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:28.304 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.323+0000 7f8404938700 1 -- 172.21.15.17:0/3660617744 <== mon.2 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f83f00ce1c0 con 0x7f84000ffb90 2023-07-14T12:01:28.304 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.323+0000 7f83fe7fc700 1 --2- 172.21.15.17:0/3660617744 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f83e80a1150 0x7f83e80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:28.304 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.324+0000 7f83fe7fc700 1 --2- 172.21.15.17:0/3660617744 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f83e80a1150 0x7f83e80a3600 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f83f4009d20 tx=0x7f83f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:28.305 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.325+0000 7f8404938700 1 -- 172.21.15.17:0/3660617744 <== mon.2 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f83f00975f0 con 0x7f84000ffb90 2023-07-14T12:01:28.522 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.542+0000 7f840593a700 1 -- 172.21.15.17:0/3660617744 --> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f8400060950 con 0x7f83e80a1150 2023-07-14T12:01:28.528 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.548+0000 7f8404938700 1 -- 172.21.15.17:0/3660617744 <== mgr.14152 v2:172.21.15.17:6800/4237449858 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f8400060950 con 0x7f83e80a1150 2023-07-14T12:01:28.528 INFO:teuthology.orchestra.run.smithi017.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-07-14T12:01:28.528 INFO:teuthology.orchestra.run.smithi017.stdout:mds.all 12/12 40s ago 2m smithi017=a;smithi017=d;smithi017=g;smithi017=j;smithi093=b;smithi093=e;smithi093=h;smithi093=k;smithi097=c;smithi097=f;smithi097=i;smithi097=l;count:12 2023-07-14T12:01:28.528 INFO:teuthology.orchestra.run.smithi017.stdout:mgr 3/3 40s ago 7m smithi017=x;smithi093=y;smithi097=z;count:3 2023-07-14T12:01:28.529 INFO:teuthology.orchestra.run.smithi017.stdout:mon 3/3 40s ago 7m smithi017:172.21.15.17=a;smithi093:172.21.15.93=b;smithi097:172.21.15.97=c;count:3 2023-07-14T12:01:28.529 INFO:teuthology.orchestra.run.smithi017.stdout:osd 12 40s ago - 2023-07-14T12:01:28.530 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.550+0000 7f840593a700 1 -- 172.21.15.17:0/3660617744 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f83e80a1150 msgr2=0x7f83e80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:28.530 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.550+0000 7f840593a700 1 --2- 172.21.15.17:0/3660617744 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f83e80a1150 0x7f83e80a3600 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f83f4009d20 tx=0x7f83f4009260 comp rx=0 tx=0).stop 2023-07-14T12:01:28.531 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.550+0000 7f840593a700 1 -- 172.21.15.17:0/3660617744 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84000ffb90 msgr2=0x7f8400197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:28.531 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.550+0000 7f840593a700 1 --2- 172.21.15.17:0/3660617744 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84000ffb90 0x7f8400197ab0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f83f00098e0 tx=0x7f83f0009580 comp rx=0 tx=0).stop 2023-07-14T12:01:28.531 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.550+0000 7f840593a700 1 -- 172.21.15.17:0/3660617744 shutdown_connections 2023-07-14T12:01:28.531 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.550+0000 7f840593a700 1 --2- 172.21.15.17:0/3660617744 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f83e80a1150 0x7f83e80a3600 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:28.531 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.550+0000 7f840593a700 1 --2- 172.21.15.17:0/3660617744 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84000ffb90 0x7f8400197ab0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:28.531 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.550+0000 7f840593a700 1 --2- 172.21.15.17:0/3660617744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8400101720 0x7f840019f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:28.531 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.550+0000 7f840593a700 1 --2- 172.21.15.17:0/3660617744 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8400100d90 0x7f8400197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:28.532 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.550+0000 7f840593a700 1 -- 172.21.15.17:0/3660617744 >> 172.21.15.17:0/3660617744 conn(0x7f84000fb1a0 msgr2=0x7f8400106720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:28.532 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.550+0000 7f840593a700 1 -- 172.21.15.17:0/3660617744 shutdown_connections 2023-07-14T12:01:28.532 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:28.550+0000 7f840593a700 1 -- 172.21.15.17:0/3660617744 wait complete. 2023-07-14T12:01:28.991 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- bash -c 'ceph orch host ls' 2023-07-14T12:01:29.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:28 smithi093 ceph-mon[128922]: pgmap v333: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-07-14T12:01:29.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:28 smithi097 ceph-mon[129313]: pgmap v333: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-07-14T12:01:29.143 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:28 smithi017 ceph-mon[126339]: pgmap v333: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-07-14T12:01:29.726 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T12:01:30.097 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:29 smithi017 ceph-mon[126339]: from='client.25187 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T12:01:30.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:29 smithi093 ceph-mon[128922]: from='client.25187 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T12:01:30.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:29 smithi097 ceph-mon[129313]: from='client.25187 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T12:01:31.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:30 smithi093 ceph-mon[128922]: pgmap v334: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-07-14T12:01:31.133 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:30 smithi097 ceph-mon[129313]: pgmap v334: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-07-14T12:01:31.143 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:30 smithi017 ceph-mon[126339]: pgmap v334: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-07-14T12:01:31.264 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.284+0000 7f61279b3700 1 -- 172.21.15.17:0/704992729 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6120101720 msgr2=0x7f6120107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:31.264 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.284+0000 7f61279b3700 1 --2- 172.21.15.17:0/704992729 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6120101720 0x7f6120107fe0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f611800b0a0 tx=0x7f611800b3b0 comp rx=0 tx=0).stop 2023-07-14T12:01:31.265 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.284+0000 7f61279b3700 1 -- 172.21.15.17:0/704992729 shutdown_connections 2023-07-14T12:01:31.265 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.284+0000 7f61279b3700 1 --2- 172.21.15.17:0/704992729 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6120101720 0x7f6120107fe0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:31.265 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.284+0000 7f61279b3700 1 --2- 172.21.15.17:0/704992729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61200ffb90 0x7f61200fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:31.265 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.284+0000 7f61279b3700 1 --2- 172.21.15.17:0/704992729 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6120100d90 0x7f61201011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:31.265 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.284+0000 7f61279b3700 1 -- 172.21.15.17:0/704992729 >> 172.21.15.17:0/704992729 conn(0x7f61200fb1a0 msgr2=0x7f61200fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:31.265 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.284+0000 7f61279b3700 1 -- 172.21.15.17:0/704992729 shutdown_connections 2023-07-14T12:01:31.266 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.284+0000 7f61279b3700 1 -- 172.21.15.17:0/704992729 wait complete. 2023-07-14T12:01:31.267 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.285+0000 7f61279b3700 1 Processor -- start 2023-07-14T12:01:31.267 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.286+0000 7f61279b3700 1 -- start start 2023-07-14T12:01:31.267 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.286+0000 7f61279b3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61200ffb90 0x7f61200742b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:31.267 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.286+0000 7f61279b3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6120100d90 0x7f6120072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:31.267 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.286+0000 7f61279b3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6120101720 0x7f6120072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:31.268 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.286+0000 7f61279b3700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61200735c0 con 0x7f6120101720 2023-07-14T12:01:31.268 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.286+0000 7f61279b3700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6120073730 con 0x7f6120100d90 2023-07-14T12:01:31.268 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.286+0000 7f61279b3700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612019fa50 con 0x7f61200ffb90 2023-07-14T12:01:31.268 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.286+0000 7f6124f4e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6120100d90 0x7f6120072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:31.268 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.286+0000 7f6125f50700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6120101720 0x7f6120072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:31.268 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.286+0000 7f6124f4e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6120100d90 0x7f6120072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.17:57108/0 (socket says 172.21.15.17:57108) 2023-07-14T12:01:31.268 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.286+0000 7f6124f4e700 1 -- 172.21.15.17:0/1256594743 learned_addr learned my addr 172.21.15.17:0/1256594743 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:31.269 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.286+0000 7f612574f700 1 --2- 172.21.15.17:0/1256594743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61200ffb90 0x7f61200742b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:31.269 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.287+0000 7f6124f4e700 1 -- 172.21.15.17:0/1256594743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61200ffb90 msgr2=0x7f61200742b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:31.269 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.287+0000 7f6124f4e700 1 --2- 172.21.15.17:0/1256594743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61200ffb90 0x7f61200742b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:31.269 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.287+0000 7f6124f4e700 1 -- 172.21.15.17:0/1256594743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6120101720 msgr2=0x7f6120072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:31.269 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.287+0000 7f6124f4e700 1 --2- 172.21.15.17:0/1256594743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6120101720 0x7f6120072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:31.269 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.287+0000 7f6124f4e700 1 -- 172.21.15.17:0/1256594743 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6118009d00 con 0x7f6120100d90 2023-07-14T12:01:31.269 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.287+0000 7f6125f50700 1 --2- 172.21.15.17:0/1256594743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6120101720 0x7f6120072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T12:01:31.269 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.287+0000 7f6124f4e700 1 --2- 172.21.15.17:0/1256594743 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6120100d90 0x7f6120072900 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f611000e9a0 tx=0x7f611000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:31.270 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.287+0000 7f61167fc700 1 -- 172.21.15.17:0/1256594743 <== mon.2 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6110009790 con 0x7f6120100d90 2023-07-14T12:01:31.270 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.287+0000 7f61279b3700 1 -- 172.21.15.17:0/1256594743 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f612019fd30 con 0x7f6120100d90 2023-07-14T12:01:31.270 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.287+0000 7f61167fc700 1 -- 172.21.15.17:0/1256594743 <== mon.2 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6110010430 con 0x7f6120100d90 2023-07-14T12:01:31.271 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.287+0000 7f61167fc700 1 -- 172.21.15.17:0/1256594743 <== mon.2 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6110018680 con 0x7f6120100d90 2023-07-14T12:01:31.271 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.287+0000 7f61279b3700 1 -- 172.21.15.17:0/1256594743 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61201a0220 con 0x7f6120100d90 2023-07-14T12:01:31.271 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.288+0000 7f61279b3700 1 -- 172.21.15.17:0/1256594743 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61201070d0 con 0x7f6120100d90 2023-07-14T12:01:31.271 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.291+0000 7f61167fc700 1 -- 172.21.15.17:0/1256594743 <== mon.2 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f61100105a0 con 0x7f6120100d90 2023-07-14T12:01:31.272 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.292+0000 7f61167fc700 1 --2- 172.21.15.17:0/1256594743 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f610c0a1150 0x7f610c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:31.272 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.292+0000 7f61167fc700 1 -- 172.21.15.17:0/1256594743 <== mon.2 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f6110014070 con 0x7f6120100d90 2023-07-14T12:01:31.272 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.292+0000 7f612574f700 1 --2- 172.21.15.17:0/1256594743 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f610c0a1150 0x7f610c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:31.272 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.293+0000 7f612574f700 1 --2- 172.21.15.17:0/1256594743 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f610c0a1150 0x7f610c0a3600 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6120100bf0 tx=0x7f611c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:31.273 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.294+0000 7f61167fc700 1 -- 172.21.15.17:0/1256594743 <== mon.2 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6110096190 con 0x7f6120100d90 2023-07-14T12:01:31.488 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.508+0000 7f61279b3700 1 -- 172.21.15.17:0/1256594743 --> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f6120060950 con 0x7f610c0a1150 2023-07-14T12:01:31.490 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.510+0000 7f61167fc700 1 -- 172.21.15.17:0/1256594743 <== mgr.14152 v2:172.21.15.17:6800/4237449858 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+187 (secure 0 0 0) 0x7f6120060950 con 0x7f610c0a1150 2023-07-14T12:01:31.490 INFO:teuthology.orchestra.run.smithi017.stdout:HOST ADDR LABELS STATUS 2023-07-14T12:01:31.490 INFO:teuthology.orchestra.run.smithi017.stdout:smithi017 172.21.15.17 2023-07-14T12:01:31.490 INFO:teuthology.orchestra.run.smithi017.stdout:smithi093 172.21.15.93 2023-07-14T12:01:31.490 INFO:teuthology.orchestra.run.smithi017.stdout:smithi097 172.21.15.97 2023-07-14T12:01:31.491 INFO:teuthology.orchestra.run.smithi017.stdout:3 hosts in cluster 2023-07-14T12:01:31.492 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.512+0000 7f61279b3700 1 -- 172.21.15.17:0/1256594743 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f610c0a1150 msgr2=0x7f610c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:31.492 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.512+0000 7f61279b3700 1 --2- 172.21.15.17:0/1256594743 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f610c0a1150 0x7f610c0a3600 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6120100bf0 tx=0x7f611c009260 comp rx=0 tx=0).stop 2023-07-14T12:01:31.492 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.512+0000 7f61279b3700 1 -- 172.21.15.17:0/1256594743 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6120100d90 msgr2=0x7f6120072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:31.493 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.512+0000 7f61279b3700 1 --2- 172.21.15.17:0/1256594743 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6120100d90 0x7f6120072900 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f611000e9a0 tx=0x7f611000ecb0 comp rx=0 tx=0).stop 2023-07-14T12:01:31.493 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.512+0000 7f61279b3700 1 -- 172.21.15.17:0/1256594743 shutdown_connections 2023-07-14T12:01:31.493 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.512+0000 7f61279b3700 1 --2- 172.21.15.17:0/1256594743 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f610c0a1150 0x7f610c0a3600 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:31.493 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.512+0000 7f61279b3700 1 --2- 172.21.15.17:0/1256594743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6120101720 0x7f6120072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:31.493 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.512+0000 7f61279b3700 1 --2- 172.21.15.17:0/1256594743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61200ffb90 0x7f61200742b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:31.493 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.512+0000 7f61279b3700 1 --2- 172.21.15.17:0/1256594743 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6120100d90 0x7f6120072900 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:31.493 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.512+0000 7f61279b3700 1 -- 172.21.15.17:0/1256594743 >> 172.21.15.17:0/1256594743 conn(0x7f61200fb1a0 msgr2=0x7f6120104210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:31.494 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.512+0000 7f61279b3700 1 -- 172.21.15.17:0/1256594743 shutdown_connections 2023-07-14T12:01:31.494 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:31.512+0000 7f61279b3700 1 -- 172.21.15.17:0/1256594743 wait complete. 2023-07-14T12:01:32.121 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- bash -c 'ceph orch device ls' 2023-07-14T12:01:32.874 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T12:01:33.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:32 smithi093 ceph-mon[128922]: from='client.25193 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T12:01:33.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:32 smithi093 ceph-mon[128922]: pgmap v335: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-07-14T12:01:33.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:32 smithi097 ceph-mon[129313]: from='client.25193 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T12:01:33.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:32 smithi097 ceph-mon[129313]: pgmap v335: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-07-14T12:01:33.143 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:32 smithi017 ceph-mon[126339]: from='client.25193 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T12:01:33.143 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:32 smithi017 ceph-mon[126339]: pgmap v335: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-07-14T12:01:34.398 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.417+0000 7faca8fa2700 1 -- 172.21.15.17:0/4134201771 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faca4101720 msgr2=0x7faca4107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:34.398 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.417+0000 7faca8fa2700 1 --2- 172.21.15.17:0/4134201771 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faca4101720 0x7faca4107fe0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fac9c00b0a0 tx=0x7fac9c00b3b0 comp rx=0 tx=0).stop 2023-07-14T12:01:34.398 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.417+0000 7faca8fa2700 1 -- 172.21.15.17:0/4134201771 shutdown_connections 2023-07-14T12:01:34.398 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.417+0000 7faca8fa2700 1 --2- 172.21.15.17:0/4134201771 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faca40ffb90 0x7faca40fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:34.398 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.417+0000 7faca8fa2700 1 --2- 172.21.15.17:0/4134201771 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faca4101720 0x7faca4107fe0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:34.399 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.417+0000 7faca8fa2700 1 --2- 172.21.15.17:0/4134201771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faca4100d90 0x7faca41011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:34.399 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.417+0000 7faca8fa2700 1 -- 172.21.15.17:0/4134201771 >> 172.21.15.17:0/4134201771 conn(0x7faca40fb1a0 msgr2=0x7faca40fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:34.399 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.417+0000 7faca8fa2700 1 -- 172.21.15.17:0/4134201771 shutdown_connections 2023-07-14T12:01:34.399 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.417+0000 7faca8fa2700 1 -- 172.21.15.17:0/4134201771 wait complete. 2023-07-14T12:01:34.401 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.418+0000 7faca8fa2700 1 Processor -- start 2023-07-14T12:01:34.401 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.419+0000 7faca8fa2700 1 -- start start 2023-07-14T12:01:34.401 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.419+0000 7faca8fa2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faca40ffb90 0x7faca4105920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:34.401 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.419+0000 7faca8fa2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faca4100d90 0x7faca4105e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:34.401 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.419+0000 7faca8fa2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faca4101720 0x7faca41024a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:34.401 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.419+0000 7faca8fa2700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca4102c40 con 0x7faca40ffb90 2023-07-14T12:01:34.401 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.419+0000 7faca8fa2700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca4102db0 con 0x7faca4101720 2023-07-14T12:01:34.402 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.419+0000 7faca8fa2700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca4102f70 con 0x7faca4100d90 2023-07-14T12:01:34.402 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.419+0000 7faca259c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faca40ffb90 0x7faca4105920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:34.402 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.419+0000 7faca259c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faca40ffb90 0x7faca4105920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43914/0 (socket says 172.21.15.17:43914) 2023-07-14T12:01:34.402 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.419+0000 7faca259c700 1 -- 172.21.15.17:0/2423366812 learned_addr learned my addr 172.21.15.17:0/2423366812 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:34.402 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.419+0000 7faca1d9b700 1 --2- 172.21.15.17:0/2423366812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faca4100d90 0x7faca4105e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:34.402 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.419+0000 7faca2d9d700 1 --2- 172.21.15.17:0/2423366812 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faca4101720 0x7faca41024a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:34.402 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.420+0000 7faca259c700 1 -- 172.21.15.17:0/2423366812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faca4100d90 msgr2=0x7faca4105e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:34.403 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.420+0000 7faca259c700 1 --2- 172.21.15.17:0/2423366812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faca4100d90 0x7faca4105e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:34.403 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.420+0000 7faca259c700 1 -- 172.21.15.17:0/2423366812 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faca4101720 msgr2=0x7faca41024a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:34.403 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.420+0000 7faca259c700 1 --2- 172.21.15.17:0/2423366812 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faca4101720 0x7faca41024a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:34.403 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.420+0000 7faca259c700 1 -- 172.21.15.17:0/2423366812 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac9c009d00 con 0x7faca40ffb90 2023-07-14T12:01:34.403 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.420+0000 7faca259c700 1 --2- 172.21.15.17:0/2423366812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faca40ffb90 0x7faca4105920 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fac9400d980 tx=0x7fac9400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:34.403 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.420+0000 7fac937fe700 1 -- 172.21.15.17:0/2423366812 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9400b950 con 0x7faca40ffb90 2023-07-14T12:01:34.403 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.420+0000 7fac937fe700 1 -- 172.21.15.17:0/2423366812 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac94007620 con 0x7faca40ffb90 2023-07-14T12:01:34.403 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.420+0000 7fac937fe700 1 -- 172.21.15.17:0/2423366812 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9400f3c0 con 0x7faca40ffb90 2023-07-14T12:01:34.404 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.420+0000 7faca8fa2700 1 -- 172.21.15.17:0/2423366812 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faca41a4100 con 0x7faca40ffb90 2023-07-14T12:01:34.404 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.420+0000 7faca8fa2700 1 -- 172.21.15.17:0/2423366812 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faca41a45d0 con 0x7faca40ffb90 2023-07-14T12:01:34.404 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.421+0000 7faca8fa2700 1 -- 172.21.15.17:0/2423366812 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faca4192240 con 0x7faca40ffb90 2023-07-14T12:01:34.404 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.424+0000 7fac937fe700 1 -- 172.21.15.17:0/2423366812 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fac94010640 con 0x7faca40ffb90 2023-07-14T12:01:34.405 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.425+0000 7fac937fe700 1 --2- 172.21.15.17:0/2423366812 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fac8c0a11d0 0x7fac8c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:34.405 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.425+0000 7fac937fe700 1 -- 172.21.15.17:0/2423366812 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fac940ccaf0 con 0x7faca40ffb90 2023-07-14T12:01:34.405 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.425+0000 7faca1d9b700 1 --2- 172.21.15.17:0/2423366812 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fac8c0a11d0 0x7fac8c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:34.406 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.426+0000 7faca1d9b700 1 --2- 172.21.15.17:0/2423366812 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fac8c0a11d0 0x7fac8c0a3680 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fac98009be0 tx=0x7fac98009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:34.409 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.429+0000 7fac937fe700 1 -- 172.21.15.17:0/2423366812 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fac94095e90 con 0x7faca40ffb90 2023-07-14T12:01:34.625 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.645+0000 7faca8fa2700 1 -- 172.21.15.17:0/2423366812 --> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7faca4060950 con 0x7fac8c0a11d0 2023-07-14T12:01:34.626 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.646+0000 7fac937fe700 1 -- 172.21.15.17:0/2423366812 <== mgr.14152 v2:172.21.15.17:6800/4237449858 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+492 (secure 0 0 0) 0x7faca4060950 con 0x7fac8c0a11d0 2023-07-14T12:01:34.627 INFO:teuthology.orchestra.run.smithi017.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-07-14T12:01:34.627 INFO:teuthology.orchestra.run.smithi017.stdout:smithi017 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7473003Q480DGN 447G No 3m ago LVM detected, locked 2023-07-14T12:01:34.627 INFO:teuthology.orchestra.run.smithi017.stdout:smithi093 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040150400BGN 372G No 3m ago LVM detected, locked 2023-07-14T12:01:34.627 INFO:teuthology.orchestra.run.smithi017.stdout:smithi097 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204013T400BGN 372G No 2m ago LVM detected, locked 2023-07-14T12:01:34.629 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.649+0000 7faca8fa2700 1 -- 172.21.15.17:0/2423366812 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fac8c0a11d0 msgr2=0x7fac8c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:34.629 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.649+0000 7faca8fa2700 1 --2- 172.21.15.17:0/2423366812 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fac8c0a11d0 0x7fac8c0a3680 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fac98009be0 tx=0x7fac98009260 comp rx=0 tx=0).stop 2023-07-14T12:01:34.630 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.649+0000 7faca8fa2700 1 -- 172.21.15.17:0/2423366812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faca40ffb90 msgr2=0x7faca4105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:34.630 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.649+0000 7faca8fa2700 1 --2- 172.21.15.17:0/2423366812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faca40ffb90 0x7faca4105920 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fac9400d980 tx=0x7fac9400dc90 comp rx=0 tx=0).stop 2023-07-14T12:01:34.630 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.649+0000 7faca8fa2700 1 -- 172.21.15.17:0/2423366812 shutdown_connections 2023-07-14T12:01:34.630 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.649+0000 7faca8fa2700 1 --2- 172.21.15.17:0/2423366812 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fac8c0a11d0 0x7fac8c0a3680 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:34.630 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.649+0000 7faca8fa2700 1 --2- 172.21.15.17:0/2423366812 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faca4101720 0x7faca41024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:34.630 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.649+0000 7faca8fa2700 1 --2- 172.21.15.17:0/2423366812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faca40ffb90 0x7faca4105920 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:34.630 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.649+0000 7faca8fa2700 1 --2- 172.21.15.17:0/2423366812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faca4100d90 0x7faca4105e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:34.631 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.649+0000 7faca8fa2700 1 -- 172.21.15.17:0/2423366812 >> 172.21.15.17:0/2423366812 conn(0x7faca40fb1a0 msgr2=0x7faca4106670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:34.631 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.649+0000 7faca8fa2700 1 -- 172.21.15.17:0/2423366812 shutdown_connections 2023-07-14T12:01:34.631 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:34.649+0000 7faca8fa2700 1 -- 172.21.15.17:0/2423366812 wait complete. 2023-07-14T12:01:34.787 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:34 smithi017 ceph-mon[126339]: pgmap v336: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-07-14T12:01:34.860 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:34 smithi093 ceph-mon[128922]: pgmap v336: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-07-14T12:01:34.884 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:34 smithi097 ceph-mon[129313]: pgmap v336: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-07-14T12:01:35.071 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-07-14T12:01:35.082 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi017.front.sepia.ceph.com 2023-07-14T12:01:35.082 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- bash -c 'ceph fs dump' 2023-07-14T12:01:35.817 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T12:01:35.860 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:35 smithi093 ceph-mon[128922]: from='client.15192 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T12:01:35.884 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:35 smithi097 ceph-mon[129313]: from='client.15192 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T12:01:35.893 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:35 smithi017 ceph-mon[126339]: from='client.15192 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T12:01:36.643 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:36 smithi017 ceph-mon[126339]: pgmap v337: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-14T12:01:36.860 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:36 smithi093 ceph-mon[128922]: pgmap v337: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-14T12:01:36.883 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:36 smithi097 ceph-mon[129313]: pgmap v337: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-14T12:01:37.406 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.427+0000 7f3d7f26b700 1 -- 172.21.15.17:0/1861477047 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d78101720 msgr2=0x7f3d78107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:37.407 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.427+0000 7f3d7f26b700 1 --2- 172.21.15.17:0/1861477047 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d78101720 0x7f3d78107fe0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3d7000b0a0 tx=0x7f3d7000b3b0 comp rx=0 tx=0).stop 2023-07-14T12:01:37.407 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.427+0000 7f3d7f26b700 1 -- 172.21.15.17:0/1861477047 shutdown_connections 2023-07-14T12:01:37.407 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.427+0000 7f3d7f26b700 1 --2- 172.21.15.17:0/1861477047 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3d780ffb90 0x7f3d780fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:37.407 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.427+0000 7f3d7f26b700 1 --2- 172.21.15.17:0/1861477047 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d78101720 0x7f3d78107fe0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:37.407 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.427+0000 7f3d7f26b700 1 --2- 172.21.15.17:0/1861477047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d78100d90 0x7f3d781011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:37.407 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.427+0000 7f3d7f26b700 1 -- 172.21.15.17:0/1861477047 >> 172.21.15.17:0/1861477047 conn(0x7f3d780fb1a0 msgr2=0x7f3d780fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:37.408 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.427+0000 7f3d7f26b700 1 -- 172.21.15.17:0/1861477047 shutdown_connections 2023-07-14T12:01:37.408 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.427+0000 7f3d7f26b700 1 -- 172.21.15.17:0/1861477047 wait complete. 2023-07-14T12:01:37.409 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.428+0000 7f3d7f26b700 1 Processor -- start 2023-07-14T12:01:37.409 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.428+0000 7f3d7f26b700 1 -- start start 2023-07-14T12:01:37.409 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.429+0000 7f3d7f26b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3d780ffb90 0x7f3d78195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:37.410 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.429+0000 7f3d7f26b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d78100d90 0x7f3d78195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:37.410 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.429+0000 7f3d7f26b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d78101720 0x7f3d7819cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:37.410 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.429+0000 7f3d7f26b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7819d500 con 0x7f3d78100d90 2023-07-14T12:01:37.410 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.429+0000 7f3d7f26b700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7819d640 con 0x7f3d780ffb90 2023-07-14T12:01:37.410 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.429+0000 7f3d7f26b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7819d7d0 con 0x7f3d78101720 2023-07-14T12:01:37.410 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.429+0000 7f3d7d007700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3d780ffb90 0x7f3d78195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:37.410 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.429+0000 7f3d7d007700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3d780ffb90 0x7f3d78195970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.17:38130/0 (socket says 172.21.15.17:38130) 2023-07-14T12:01:37.411 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.429+0000 7f3d7d007700 1 -- 172.21.15.17:0/3456515223 learned_addr learned my addr 172.21.15.17:0/3456515223 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:37.411 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.429+0000 7f3d7c806700 1 --2- 172.21.15.17:0/3456515223 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d78100d90 0x7f3d78195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:37.411 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.429+0000 7f3d7d808700 1 --2- 172.21.15.17:0/3456515223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d78101720 0x7f3d7819cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:37.411 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.429+0000 7f3d7d007700 1 -- 172.21.15.17:0/3456515223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d78101720 msgr2=0x7f3d7819cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:37.411 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.429+0000 7f3d7d007700 1 --2- 172.21.15.17:0/3456515223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d78101720 0x7f3d7819cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:37.411 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.429+0000 7f3d7d007700 1 -- 172.21.15.17:0/3456515223 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d78100d90 msgr2=0x7f3d78195eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:37.411 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.429+0000 7f3d7d007700 1 --2- 172.21.15.17:0/3456515223 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d78100d90 0x7f3d78195eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:37.411 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.429+0000 7f3d7d007700 1 -- 172.21.15.17:0/3456515223 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d70009d00 con 0x7f3d780ffb90 2023-07-14T12:01:37.412 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.430+0000 7f3d7d808700 1 --2- 172.21.15.17:0/3456515223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d78101720 0x7f3d7819cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T12:01:37.412 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.430+0000 7f3d7c806700 1 --2- 172.21.15.17:0/3456515223 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d78100d90 0x7f3d78195eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T12:01:37.412 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.430+0000 7f3d7d007700 1 --2- 172.21.15.17:0/3456515223 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3d780ffb90 0x7f3d78195970 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3d7400d980 tx=0x7f3d7400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:37.412 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.430+0000 7f3d6e7fc700 1 -- 172.21.15.17:0/3456515223 <== mon.2 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d74004020 con 0x7f3d780ffb90 2023-07-14T12:01:37.412 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.430+0000 7f3d7f26b700 1 -- 172.21.15.17:0/3456515223 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d7819dab0 con 0x7f3d780ffb90 2023-07-14T12:01:37.412 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.430+0000 7f3d6e7fc700 1 -- 172.21.15.17:0/3456515223 <== mon.2 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d7400f690 con 0x7f3d780ffb90 2023-07-14T12:01:37.412 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.430+0000 7f3d6e7fc700 1 -- 172.21.15.17:0/3456515223 <== mon.2 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d74003a80 con 0x7f3d780ffb90 2023-07-14T12:01:37.413 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.430+0000 7f3d7f26b700 1 -- 172.21.15.17:0/3456515223 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d7819e000 con 0x7f3d780ffb90 2023-07-14T12:01:37.413 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.431+0000 7f3d7f26b700 1 -- 172.21.15.17:0/3456515223 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d78107130 con 0x7f3d780ffb90 2023-07-14T12:01:37.413 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.433+0000 7f3d6e7fc700 1 -- 172.21.15.17:0/3456515223 <== mon.2 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3d7400f800 con 0x7f3d780ffb90 2023-07-14T12:01:37.414 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.435+0000 7f3d6e7fc700 1 --2- 172.21.15.17:0/3456515223 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3d640a1150 0x7f3d640a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:37.414 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.435+0000 7f3d6e7fc700 1 -- 172.21.15.17:0/3456515223 <== mon.2 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f3d740cc820 con 0x7f3d780ffb90 2023-07-14T12:01:37.415 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.435+0000 7f3d7c806700 1 --2- 172.21.15.17:0/3456515223 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3d640a1150 0x7f3d640a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:37.415 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.436+0000 7f3d7c806700 1 --2- 172.21.15.17:0/3456515223 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3d640a1150 0x7f3d640a3600 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3d68011490 tx=0x7f3d68009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:37.416 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.437+0000 7f3d6e7fc700 1 -- 172.21.15.17:0/3456515223 <== mon.2 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3d74095bc0 con 0x7f3d780ffb90 2023-07-14T12:01:37.665 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.686+0000 7f3d7f26b700 1 -- 172.21.15.17:0/3456515223 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3d7802c850 con 0x7f3d780ffb90 2023-07-14T12:01:37.666 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.687+0000 7f3d6e7fc700 1 -- 172.21.15.17:0/3456515223 <== mon.2 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) v1 ==== 76+0+2913 (secure 0 0 0) 0x7f3d740991e0 con 0x7f3d780ffb90 2023-07-14T12:01:37.666 INFO:teuthology.orchestra.run.smithi017.stdout:e27 2023-07-14T12:01:37.667 INFO:teuthology.orchestra.run.smithi017.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-07-14T12:01:37.667 INFO:teuthology.orchestra.run.smithi017.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-14T12:01:37.667 INFO:teuthology.orchestra.run.smithi017.stdout:legacy client fscid: 1 2023-07-14T12:01:37.667 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T12:01:37.667 INFO:teuthology.orchestra.run.smithi017.stdout:Filesystem 'cephfs' (1) 2023-07-14T12:01:37.667 INFO:teuthology.orchestra.run.smithi017.stdout:fs_name cephfs 2023-07-14T12:01:37.667 INFO:teuthology.orchestra.run.smithi017.stdout:epoch 27 2023-07-14T12:01:37.668 INFO:teuthology.orchestra.run.smithi017.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-07-14T12:01:37.668 INFO:teuthology.orchestra.run.smithi017.stdout:created 2023-07-14T11:59:43.292866+0000 2023-07-14T12:01:37.668 INFO:teuthology.orchestra.run.smithi017.stdout:modified 2023-07-14T12:00:17.489445+0000 2023-07-14T12:01:37.668 INFO:teuthology.orchestra.run.smithi017.stdout:tableserver 0 2023-07-14T12:01:37.668 INFO:teuthology.orchestra.run.smithi017.stdout:root 0 2023-07-14T12:01:37.668 INFO:teuthology.orchestra.run.smithi017.stdout:session_timeout 300 2023-07-14T12:01:37.668 INFO:teuthology.orchestra.run.smithi017.stdout:session_autoclose 300 2023-07-14T12:01:37.669 INFO:teuthology.orchestra.run.smithi017.stdout:max_file_size 1099511627776 2023-07-14T12:01:37.669 INFO:teuthology.orchestra.run.smithi017.stdout:max_xattr_size 65536 2023-07-14T12:01:37.669 INFO:teuthology.orchestra.run.smithi017.stdout:required_client_features {} 2023-07-14T12:01:37.669 INFO:teuthology.orchestra.run.smithi017.stdout:last_failure 0 2023-07-14T12:01:37.669 INFO:teuthology.orchestra.run.smithi017.stdout:last_failure_osd_epoch 0 2023-07-14T12:01:37.669 INFO:teuthology.orchestra.run.smithi017.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-14T12:01:37.669 INFO:teuthology.orchestra.run.smithi017.stdout:max_mds 5 2023-07-14T12:01:37.669 INFO:teuthology.orchestra.run.smithi017.stdout:in 0,1,2,3,4 2023-07-14T12:01:37.670 INFO:teuthology.orchestra.run.smithi017.stdout:up {0=24565,1=24557,2=24541,3=24533,4=24517} 2023-07-14T12:01:37.670 INFO:teuthology.orchestra.run.smithi017.stdout:failed 2023-07-14T12:01:37.670 INFO:teuthology.orchestra.run.smithi017.stdout:damaged 2023-07-14T12:01:37.670 INFO:teuthology.orchestra.run.smithi017.stdout:stopped 2023-07-14T12:01:37.670 INFO:teuthology.orchestra.run.smithi017.stdout:data_pools [3,4] 2023-07-14T12:01:37.670 INFO:teuthology.orchestra.run.smithi017.stdout:metadata_pool 2 2023-07-14T12:01:37.670 INFO:teuthology.orchestra.run.smithi017.stdout:inline_data disabled 2023-07-14T12:01:37.670 INFO:teuthology.orchestra.run.smithi017.stdout:balancer 2023-07-14T12:01:37.671 INFO:teuthology.orchestra.run.smithi017.stdout:bal_rank_mask -1 2023-07-14T12:01:37.671 INFO:teuthology.orchestra.run.smithi017.stdout:standby_count_wanted 0 2023-07-14T12:01:37.671 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.a{0:24565} state up:active seq 4 addr [v2:172.21.15.17:6840/3905201143,v1:172.21.15.17:6841/3905201143] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T12:01:37.671 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.h{0:24509} state up:standby-replay seq 1 addr [v2:172.21.15.93:6836/1880673911,v1:172.21.15.93:6837/1880673911] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T12:01:37.671 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.b{1:24557} state up:active seq 10 addr [v2:172.21.15.93:6838/1281342428,v1:172.21.15.93:6839/1281342428] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T12:01:37.671 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.f{1:24505} state up:standby-replay seq 1 addr [v2:172.21.15.97:6834/2740799393,v1:172.21.15.97:6835/2740799393] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T12:01:37.671 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.i{2:24541} state up:active seq 13 addr [v2:172.21.15.97:6838/3267815331,v1:172.21.15.97:6839/3267815331] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T12:01:37.671 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.e{2:24482} state up:standby-replay seq 1 addr [v2:172.21.15.93:6834/2779420503,v1:172.21.15.93:6835/2779420503] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T12:01:37.672 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.g{3:24533} state up:active seq 12 addr [v2:172.21.15.17:6838/2976493015,v1:172.21.15.17:6839/2976493015] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T12:01:37.672 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.l{3:24454} state up:standby-replay seq 1 addr [v2:172.21.15.97:6832/878913679,v1:172.21.15.97:6833/878913679] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T12:01:37.672 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.c{4:24517} state up:active seq 15 addr [v2:172.21.15.97:6836/3684187131,v1:172.21.15.97:6837/3684187131] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T12:01:37.672 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.k{4:24452} state up:standby-replay seq 1 addr [v2:172.21.15.93:6832/326319712,v1:172.21.15.93:6833/326319712] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T12:01:37.672 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T12:01:37.672 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T12:01:37.672 INFO:teuthology.orchestra.run.smithi017.stdout:Standby daemons: 2023-07-14T12:01:37.673 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T12:01:37.673 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.j{-1:14472} state up:standby seq 1 addr [v2:172.21.15.17:6834/55384193,v1:172.21.15.17:6835/55384193] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T12:01:37.673 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.d{-1:14484} state up:standby seq 1 addr [v2:172.21.15.17:6836/424956554,v1:172.21.15.17:6837/424956554] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T12:01:37.673 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.689+0000 7f3d7f26b700 1 -- 172.21.15.17:0/3456515223 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3d640a1150 msgr2=0x7f3d640a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:37.674 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.690+0000 7f3d7f26b700 1 --2- 172.21.15.17:0/3456515223 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3d640a1150 0x7f3d640a3600 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3d68011490 tx=0x7f3d68009260 comp rx=0 tx=0).stop 2023-07-14T12:01:37.674 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.690+0000 7f3d7f26b700 1 -- 172.21.15.17:0/3456515223 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3d780ffb90 msgr2=0x7f3d78195970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:37.674 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.690+0000 7f3d7f26b700 1 --2- 172.21.15.17:0/3456515223 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3d780ffb90 0x7f3d78195970 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3d7400d980 tx=0x7f3d7400dc90 comp rx=0 tx=0).stop 2023-07-14T12:01:37.674 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.690+0000 7f3d7f26b700 1 -- 172.21.15.17:0/3456515223 shutdown_connections 2023-07-14T12:01:37.674 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.690+0000 7f3d7f26b700 1 --2- 172.21.15.17:0/3456515223 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f3d640a1150 0x7f3d640a3600 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:37.674 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.690+0000 7f3d7f26b700 1 --2- 172.21.15.17:0/3456515223 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3d780ffb90 0x7f3d78195970 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:37.675 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.690+0000 7f3d7f26b700 1 --2- 172.21.15.17:0/3456515223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d78101720 0x7f3d7819cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:37.675 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.690+0000 7f3d7f26b700 1 --2- 172.21.15.17:0/3456515223 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d78100d90 0x7f3d78195eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:37.675 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.690+0000 7f3d7f26b700 1 -- 172.21.15.17:0/3456515223 >> 172.21.15.17:0/3456515223 conn(0x7f3d780fb1a0 msgr2=0x7f3d78104270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:37.675 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.690+0000 7f3d7f26b700 1 -- 172.21.15.17:0/3456515223 shutdown_connections 2023-07-14T12:01:37.675 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:37.690+0000 7f3d7f26b700 1 -- 172.21.15.17:0/3456515223 wait complete. 2023-07-14T12:01:37.675 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 27 2023-07-14T12:01:38.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:37 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/3456515223' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-07-14T12:01:38.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:37 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/3456515223' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-07-14T12:01:38.141 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:37 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/3456515223' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-07-14T12:01:38.209 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c386cc-223c-11ee-9b33-001a4aab830c -- bash -c 'ceph osd dump' 2023-07-14T12:01:38.942 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T12:01:39.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:38 smithi093 ceph-mon[128922]: pgmap v338: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-07-14T12:01:39.133 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:38 smithi097 ceph-mon[129313]: pgmap v338: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-07-14T12:01:39.143 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:38 smithi017 ceph-mon[126339]: pgmap v338: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-07-14T12:01:40.470 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.490+0000 7f25b7c5b700 1 -- 172.21.15.17:0/824381410 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25b0101720 msgr2=0x7f25b0107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:40.470 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.490+0000 7f25b7c5b700 1 --2- 172.21.15.17:0/824381410 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25b0101720 0x7f25b0107fe0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f25a800b0a0 tx=0x7f25a800b3b0 comp rx=0 tx=0).stop 2023-07-14T12:01:40.470 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.490+0000 7f25b7c5b700 1 -- 172.21.15.17:0/824381410 shutdown_connections 2023-07-14T12:01:40.470 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.490+0000 7f25b7c5b700 1 --2- 172.21.15.17:0/824381410 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25b0101720 0x7f25b0107fe0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:40.470 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.490+0000 7f25b7c5b700 1 --2- 172.21.15.17:0/824381410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25b00ffb90 0x7f25b00fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:40.470 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.490+0000 7f25b7c5b700 1 --2- 172.21.15.17:0/824381410 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f25b0100d90 0x7f25b01011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:40.471 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.490+0000 7f25b7c5b700 1 -- 172.21.15.17:0/824381410 >> 172.21.15.17:0/824381410 conn(0x7f25b00fb1a0 msgr2=0x7f25b00fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:40.471 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.490+0000 7f25b7c5b700 1 -- 172.21.15.17:0/824381410 shutdown_connections 2023-07-14T12:01:40.471 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.490+0000 7f25b7c5b700 1 -- 172.21.15.17:0/824381410 wait complete. 2023-07-14T12:01:40.472 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.492+0000 7f25b7c5b700 1 Processor -- start 2023-07-14T12:01:40.472 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.492+0000 7f25b7c5b700 1 -- start start 2023-07-14T12:01:40.473 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.492+0000 7f25b7c5b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f25b00ffb90 0x7f25b0197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:40.473 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.492+0000 7f25b7c5b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25b0100d90 0x7f25b01980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:40.473 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.492+0000 7f25b7c5b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25b0101720 0x7f25b019f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:40.473 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.492+0000 7f25b7c5b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b019f740 con 0x7f25b0101720 2023-07-14T12:01:40.473 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.492+0000 7f25b7c5b700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b019f880 con 0x7f25b00ffb90 2023-07-14T12:01:40.473 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.492+0000 7f25b7c5b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b019fa10 con 0x7f25b0100d90 2023-07-14T12:01:40.473 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.492+0000 7f25b61f8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25b0101720 0x7f25b019f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:40.473 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.492+0000 7f25b59f7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f25b00ffb90 0x7f25b0197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:40.474 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.492+0000 7f25b61f8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25b0101720 0x7f25b019f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43964/0 (socket says 172.21.15.17:43964) 2023-07-14T12:01:40.474 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.492+0000 7f25b61f8700 1 -- 172.21.15.17:0/3394239547 learned_addr learned my addr 172.21.15.17:0/3394239547 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:40.474 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.493+0000 7f25b51f6700 1 --2- 172.21.15.17:0/3394239547 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25b0100d90 0x7f25b01980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:40.474 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.493+0000 7f25b61f8700 1 -- 172.21.15.17:0/3394239547 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25b0100d90 msgr2=0x7f25b01980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:40.474 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.493+0000 7f25b61f8700 1 --2- 172.21.15.17:0/3394239547 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25b0100d90 0x7f25b01980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:40.474 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.493+0000 7f25b61f8700 1 -- 172.21.15.17:0/3394239547 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f25b00ffb90 msgr2=0x7f25b0197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:40.474 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.493+0000 7f25b61f8700 1 --2- 172.21.15.17:0/3394239547 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f25b00ffb90 0x7f25b0197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:40.475 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.493+0000 7f25b61f8700 1 -- 172.21.15.17:0/3394239547 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a8009d00 con 0x7f25b0101720 2023-07-14T12:01:40.475 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.493+0000 7f25b61f8700 1 --2- 172.21.15.17:0/3394239547 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25b0101720 0x7f25b019f170 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f25a800b070 tx=0x7f25a800bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:40.475 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.493+0000 7f25a6ffd700 1 -- 172.21.15.17:0/3394239547 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a800e040 con 0x7f25b0101720 2023-07-14T12:01:40.475 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.493+0000 7f25b7c5b700 1 -- 172.21.15.17:0/3394239547 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25b019fc90 con 0x7f25b0101720 2023-07-14T12:01:40.475 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.493+0000 7f25a6ffd700 1 -- 172.21.15.17:0/3394239547 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25a80094a0 con 0x7f25b0101720 2023-07-14T12:01:40.475 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.493+0000 7f25a6ffd700 1 -- 172.21.15.17:0/3394239547 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a801b6c0 con 0x7f25b0101720 2023-07-14T12:01:40.476 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.494+0000 7f25b7c5b700 1 -- 172.21.15.17:0/3394239547 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25b01a0100 con 0x7f25b0101720 2023-07-14T12:01:40.476 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.495+0000 7f25b7c5b700 1 -- 172.21.15.17:0/3394239547 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25b0192270 con 0x7f25b0101720 2023-07-14T12:01:40.476 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.497+0000 7f25a6ffd700 1 -- 172.21.15.17:0/3394239547 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f25a801d470 con 0x7f25b0101720 2023-07-14T12:01:40.477 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.498+0000 7f25a6ffd700 1 --2- 172.21.15.17:0/3394239547 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f259c0a1150 0x7f259c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:40.477 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.499+0000 7f25a6ffd700 1 -- 172.21.15.17:0/3394239547 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f25a80cdc40 con 0x7f25b0101720 2023-07-14T12:01:40.478 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.499+0000 7f25b59f7700 1 --2- 172.21.15.17:0/3394239547 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f259c0a1150 0x7f259c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:40.478 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.499+0000 7f25b59f7700 1 --2- 172.21.15.17:0/3394239547 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f259c0a1150 0x7f259c0a3600 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f25b0100bf0 tx=0x7f25ac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:40.481 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.503+0000 7f25a6ffd700 1 -- 172.21.15.17:0/3394239547 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f25a8097070 con 0x7f25b0101720 2023-07-14T12:01:40.689 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.710+0000 7f25b7c5b700 1 -- 172.21.15.17:0/3394239547 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f25b0061120 con 0x7f25b0101720 2023-07-14T12:01:40.690 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.711+0000 7f25a6ffd700 1 -- 172.21.15.17:0/3394239547 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v92) v1 ==== 56+0+5622 (secure 0 0 0) 0x7f25a809a690 con 0x7f25b0101720 2023-07-14T12:01:40.690 INFO:teuthology.orchestra.run.smithi017.stdout:epoch 92 2023-07-14T12:01:40.690 INFO:teuthology.orchestra.run.smithi017.stdout:fsid b3c386cc-223c-11ee-9b33-001a4aab830c 2023-07-14T12:01:40.691 INFO:teuthology.orchestra.run.smithi017.stdout:created 2023-07-14T11:51:59.992960+0000 2023-07-14T12:01:40.691 INFO:teuthology.orchestra.run.smithi017.stdout:modified 2023-07-14T12:01:07.806062+0000 2023-07-14T12:01:40.691 INFO:teuthology.orchestra.run.smithi017.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-07-14T12:01:40.691 INFO:teuthology.orchestra.run.smithi017.stdout:crush_version 27 2023-07-14T12:01:40.691 INFO:teuthology.orchestra.run.smithi017.stdout:full_ratio 0.95 2023-07-14T12:01:40.691 INFO:teuthology.orchestra.run.smithi017.stdout:backfillfull_ratio 0.9 2023-07-14T12:01:40.691 INFO:teuthology.orchestra.run.smithi017.stdout:nearfull_ratio 0.85 2023-07-14T12:01:40.691 INFO:teuthology.orchestra.run.smithi017.stdout:require_min_compat_client luminous 2023-07-14T12:01:40.692 INFO:teuthology.orchestra.run.smithi017.stdout:min_compat_client luminous 2023-07-14T12:01:40.692 INFO:teuthology.orchestra.run.smithi017.stdout:require_osd_release reef 2023-07-14T12:01:40.692 INFO:teuthology.orchestra.run.smithi017.stdout:stretch_mode_enabled false 2023-07-14T12:01:40.692 INFO:teuthology.orchestra.run.smithi017.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-07-14T12:01:40.692 INFO:teuthology.orchestra.run.smithi017.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 83 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2023-07-14T12:01:40.692 INFO:teuthology.orchestra.run.smithi017.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2023-07-14T12:01:40.692 INFO:teuthology.orchestra.run.smithi017.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 88 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2023-07-14T12:01:40.693 INFO:teuthology.orchestra.run.smithi017.stdout:max_osd 12 2023-07-14T12:01:40.693 INFO:teuthology.orchestra.run.smithi017.stdout:osd.0 up in weight 1 up_from 8 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.17:6802/3588366384,v1:172.21.15.17:6803/3588366384] [v2:172.21.15.17:6804/3588366384,v1:172.21.15.17:6805/3588366384] exists,up 57003438-a83c-4c8e-a029-56414740895c 2023-07-14T12:01:40.693 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1 up in weight 1 up_from 13 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.93:6800/3147203064,v1:172.21.15.93:6801/3147203064] [v2:172.21.15.93:6802/3147203064,v1:172.21.15.93:6803/3147203064] exists,up f56b6337-d8a9-4411-b9d8-b6863dc98556 2023-07-14T12:01:40.693 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2 up in weight 1 up_from 18 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6800/1877269815,v1:172.21.15.97:6801/1877269815] [v2:172.21.15.97:6802/1877269815,v1:172.21.15.97:6803/1877269815] exists,up fc34bb7f-551f-46e1-99c0-59924d6c1346 2023-07-14T12:01:40.693 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3 up in weight 1 up_from 26 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.17:6810/2784619821,v1:172.21.15.17:6811/2784619821] [v2:172.21.15.17:6812/2784619821,v1:172.21.15.17:6813/2784619821] exists,up 37a1583c-4fb8-412a-a0c8-f83b1ff26a91 2023-07-14T12:01:40.693 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4 up in weight 1 up_from 34 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.93:6808/3151801173,v1:172.21.15.93:6809/3151801173] [v2:172.21.15.93:6810/3151801173,v1:172.21.15.93:6811/3151801173] exists,up f4b9f377-d519-4b14-ac4a-f1debaeed9f0 2023-07-14T12:01:40.693 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5 up in weight 1 up_from 39 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6808/2184413924,v1:172.21.15.97:6809/2184413924] [v2:172.21.15.97:6810/2184413924,v1:172.21.15.97:6811/2184413924] exists,up 2d2af354-0535-4ef9-b166-3ef219498db8 2023-07-14T12:01:40.693 INFO:teuthology.orchestra.run.smithi017.stdout:osd.6 up in weight 1 up_from 47 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.17:6818/50486581,v1:172.21.15.17:6819/50486581] [v2:172.21.15.17:6820/50486581,v1:172.21.15.17:6821/50486581] exists,up 9cdf0c4b-62dc-4168-864e-a9d1650a9ea8 2023-07-14T12:01:40.694 INFO:teuthology.orchestra.run.smithi017.stdout:osd.7 up in weight 1 up_from 52 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.93:6816/2596912392,v1:172.21.15.93:6817/2596912392] [v2:172.21.15.93:6818/2596912392,v1:172.21.15.93:6819/2596912392] exists,up 09f60823-5491-45cd-9324-6ae0d56ce262 2023-07-14T12:01:40.694 INFO:teuthology.orchestra.run.smithi017.stdout:osd.8 up in weight 1 up_from 59 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6816/1965492859,v1:172.21.15.97:6817/1965492859] [v2:172.21.15.97:6818/1965492859,v1:172.21.15.97:6819/1965492859] exists,up 8c5771c9-718e-4313-a2b6-5e1c37c53f30 2023-07-14T12:01:40.694 INFO:teuthology.orchestra.run.smithi017.stdout:osd.9 up in weight 1 up_from 67 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.17:6826/2512645700,v1:172.21.15.17:6827/2512645700] [v2:172.21.15.17:6828/2512645700,v1:172.21.15.17:6829/2512645700] exists,up 1197b8a3-c09f-4c57-aba1-9e988a1620b4 2023-07-14T12:01:40.694 INFO:teuthology.orchestra.run.smithi017.stdout:osd.10 up in weight 1 up_from 72 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.93:6824/264429558,v1:172.21.15.93:6825/264429558] [v2:172.21.15.93:6826/264429558,v1:172.21.15.93:6827/264429558] exists,up f99918c2-cdd8-4a7a-be2e-d1d6e0f828a4 2023-07-14T12:01:40.694 INFO:teuthology.orchestra.run.smithi017.stdout:osd.11 up in weight 1 up_from 79 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6824/3389913754,v1:172.21.15.97:6825/3389913754] [v2:172.21.15.97:6826/3389913754,v1:172.21.15.97:6827/3389913754] exists,up 6bbb5b79-8d4d-441f-9583-203df6c78cea 2023-07-14T12:01:40.694 INFO:teuthology.orchestra.run.smithi017.stdout:pg_upmap_items 2.5 [8,11] 2023-07-14T12:01:40.694 INFO:teuthology.orchestra.run.smithi017.stdout:pg_upmap_items 2.a [2,11] 2023-07-14T12:01:40.694 INFO:teuthology.orchestra.run.smithi017.stdout:pg_upmap_items 3.1 [4,9] 2023-07-14T12:01:40.695 INFO:teuthology.orchestra.run.smithi017.stdout:pg_upmap_items 3.6 [3,7] 2023-07-14T12:01:40.695 INFO:teuthology.orchestra.run.smithi017.stdout:pg_upmap_items 3.1a [5,6] 2023-07-14T12:01:40.695 INFO:teuthology.orchestra.run.smithi017.stdout:pg_upmap_items 3.27 [4,8] 2023-07-14T12:01:40.695 INFO:teuthology.orchestra.run.smithi017.stdout:pg_upmap_items 3.2c [4,9] 2023-07-14T12:01:40.695 INFO:teuthology.orchestra.run.smithi017.stdout:pg_upmap_items 3.38 [3,9] 2023-07-14T12:01:40.695 INFO:teuthology.orchestra.run.smithi017.stdout:pg_upmap_items 4.6 [9,11] 2023-07-14T12:01:40.695 INFO:teuthology.orchestra.run.smithi017.stdout:pg_upmap_items 4.9 [5,10] 2023-07-14T12:01:40.695 INFO:teuthology.orchestra.run.smithi017.stdout:pg_upmap_items 4.13 [9,8] 2023-07-14T12:01:40.696 INFO:teuthology.orchestra.run.smithi017.stdout:pg_upmap_items 4.1b [9,8] 2023-07-14T12:01:40.696 INFO:teuthology.orchestra.run.smithi017.stdout:blocklist 172.21.15.17:0/2084033276 expires 2023-07-15T11:53:05.639597+0000 2023-07-14T12:01:40.696 INFO:teuthology.orchestra.run.smithi017.stdout:blocklist 172.21.15.17:6801/1059284716 expires 2023-07-15T11:53:05.639597+0000 2023-07-14T12:01:40.696 INFO:teuthology.orchestra.run.smithi017.stdout:blocklist 172.21.15.17:0/3651785636 expires 2023-07-15T11:53:05.639597+0000 2023-07-14T12:01:40.696 INFO:teuthology.orchestra.run.smithi017.stdout:blocklist 172.21.15.17:0/1373538024 expires 2023-07-15T11:52:37.887946+0000 2023-07-14T12:01:40.696 INFO:teuthology.orchestra.run.smithi017.stdout:blocklist 172.21.15.17:0/3560395020 expires 2023-07-15T11:52:37.887946+0000 2023-07-14T12:01:40.696 INFO:teuthology.orchestra.run.smithi017.stdout:blocklist 172.21.15.17:6800/1059284716 expires 2023-07-15T11:53:05.639597+0000 2023-07-14T12:01:40.696 INFO:teuthology.orchestra.run.smithi017.stdout:blocklist 172.21.15.17:0/4262912861 expires 2023-07-15T11:53:05.639597+0000 2023-07-14T12:01:40.697 INFO:teuthology.orchestra.run.smithi017.stdout:blocklist 172.21.15.17:0/2258375202 expires 2023-07-15T11:52:37.887946+0000 2023-07-14T12:01:40.697 INFO:teuthology.orchestra.run.smithi017.stdout:blocklist 172.21.15.17:6800/2804227742 expires 2023-07-15T11:52:37.887946+0000 2023-07-14T12:01:40.697 INFO:teuthology.orchestra.run.smithi017.stdout:blocklist 172.21.15.17:6801/2804227742 expires 2023-07-15T11:52:37.887946+0000 2023-07-14T12:01:40.697 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.713+0000 7f25b7c5b700 1 -- 172.21.15.17:0/3394239547 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f259c0a1150 msgr2=0x7f259c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:40.698 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.713+0000 7f25b7c5b700 1 --2- 172.21.15.17:0/3394239547 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f259c0a1150 0x7f259c0a3600 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f25b0100bf0 tx=0x7f25ac009260 comp rx=0 tx=0).stop 2023-07-14T12:01:40.698 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.713+0000 7f25b7c5b700 1 -- 172.21.15.17:0/3394239547 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25b0101720 msgr2=0x7f25b019f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:40.698 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.713+0000 7f25b7c5b700 1 --2- 172.21.15.17:0/3394239547 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25b0101720 0x7f25b019f170 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f25a800b070 tx=0x7f25a800bce0 comp rx=0 tx=0).stop 2023-07-14T12:01:40.698 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.714+0000 7f25b7c5b700 1 -- 172.21.15.17:0/3394239547 shutdown_connections 2023-07-14T12:01:40.698 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.714+0000 7f25b7c5b700 1 --2- 172.21.15.17:0/3394239547 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f259c0a1150 0x7f259c0a3600 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:40.698 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.714+0000 7f25b7c5b700 1 --2- 172.21.15.17:0/3394239547 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f25b00ffb90 0x7f25b0197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:40.698 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.714+0000 7f25b7c5b700 1 --2- 172.21.15.17:0/3394239547 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25b0101720 0x7f25b019f170 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:40.699 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.714+0000 7f25b7c5b700 1 --2- 172.21.15.17:0/3394239547 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25b0100d90 0x7f25b01980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:40.699 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.714+0000 7f25b7c5b700 1 -- 172.21.15.17:0/3394239547 >> 172.21.15.17:0/3394239547 conn(0x7f25b00fb1a0 msgr2=0x7f25b0106670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:40.699 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.714+0000 7f25b7c5b700 1 -- 172.21.15.17:0/3394239547 shutdown_connections 2023-07-14T12:01:40.699 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:40.714+0000 7f25b7c5b700 1 -- 172.21.15.17:0/3394239547 wait complete. 2023-07-14T12:01:41.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:40 smithi093 ceph-mon[128922]: pgmap v339: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-07-14T12:01:41.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:40 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/3394239547' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-07-14T12:01:41.133 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:40 smithi097 ceph-mon[129313]: pgmap v339: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-07-14T12:01:41.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:40 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/3394239547' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-07-14T12:01:41.143 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:40 smithi017 ceph-mon[126339]: pgmap v339: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-07-14T12:01:41.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:40 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/3394239547' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-07-14T12:01:41.240 INFO:teuthology.run_tasks:Running task fs.ready... 2023-07-14T12:01:41.250 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/run_tasks.py", line 83, in run_one_task task = get_task(taskname) File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/run_tasks.py", line 36, in get_task module = _import('tasks', module_name, task_name, fail_on_import_error=True) File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/run_tasks.py", line 55, in _import module = __import__( ModuleNotFoundError: No module named 'tasks.fs' 2023-07-14T12:01:41.317 ERROR:teuthology.run_tasks: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=b67d6784b249433495ff9d77990c5452 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/run_tasks.py", line 83, in run_one_task task = get_task(taskname) File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/run_tasks.py", line 36, in get_task module = _import('tasks', module_name, task_name, fail_on_import_error=True) File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/run_tasks.py", line 55, in _import module = __import__( ModuleNotFoundError: No module named 'tasks.fs' 2023-07-14T12:01:41.319 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-07-14T12:01:41.328 INFO:tasks.cephadm:Teardown begin 2023-07-14T12:01:41.328 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-14T12:01:41.329 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T12:01:41.542 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.563+0000 7f1a5ffff700 1 -- 172.21.15.17:0/817635076 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a580ffba0 msgr2=0x7f1a580fffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:41.543 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.563+0000 7f1a5ffff700 1 --2- 172.21.15.17:0/817635076 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a580ffba0 0x7f1a580fffb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1a54009990 tx=0x7f1a54009ca0 comp rx=0 tx=0).stop 2023-07-14T12:01:41.543 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.563+0000 7f1a5ffff700 1 -- 172.21.15.17:0/817635076 shutdown_connections 2023-07-14T12:01:41.544 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.563+0000 7f1a5ffff700 1 --2- 172.21.15.17:0/817635076 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1a581017e0 0x7f1a58108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:41.544 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.563+0000 7f1a5ffff700 1 --2- 172.21.15.17:0/817635076 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a580ffba0 0x7f1a580fffb0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:41.544 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.563+0000 7f1a5ffff700 1 --2- 172.21.15.17:0/817635076 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a58100e30 0x7f1a581012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:41.544 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.563+0000 7f1a5ffff700 1 -- 172.21.15.17:0/817635076 >> 172.21.15.17:0/817635076 conn(0x7f1a580fb1c0 msgr2=0x7f1a580fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:41.544 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.563+0000 7f1a5ffff700 1 -- 172.21.15.17:0/817635076 shutdown_connections 2023-07-14T12:01:41.545 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.563+0000 7f1a5ffff700 1 -- 172.21.15.17:0/817635076 wait complete. 2023-07-14T12:01:41.546 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.564+0000 7f1a5ffff700 1 Processor -- start 2023-07-14T12:01:41.547 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.564+0000 7f1a5ffff700 1 -- start start 2023-07-14T12:01:41.547 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.565+0000 7f1a5ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1a580ffba0 0x7f1a580742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:41.547 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.565+0000 7f1a5ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a58100e30 0x7f1a58072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:41.547 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.565+0000 7f1a5ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a581017e0 0x7f1a58072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:41.548 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.565+0000 7f1a5ffff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a580735e0 con 0x7f1a581017e0 2023-07-14T12:01:41.548 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.565+0000 7f1a5ffff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a58073750 con 0x7f1a580ffba0 2023-07-14T12:01:41.548 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.565+0000 7f1a5ffff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5819f8f0 con 0x7f1a58100e30 2023-07-14T12:01:41.548 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.565+0000 7f1a5e59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a581017e0 0x7f1a58072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:41.548 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.565+0000 7f1a5dd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1a580ffba0 0x7f1a580742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:41.549 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.565+0000 7f1a5e59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a581017e0 0x7f1a58072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43994/0 (socket says 172.21.15.17:43994) 2023-07-14T12:01:41.549 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.565+0000 7f1a5e59c700 1 -- 172.21.15.17:0/255640291 learned_addr learned my addr 172.21.15.17:0/255640291 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:41.549 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.565+0000 7f1a5d59a700 1 --2- 172.21.15.17:0/255640291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a58100e30 0x7f1a58072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:41.549 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.565+0000 7f1a5e59c700 1 -- 172.21.15.17:0/255640291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a58100e30 msgr2=0x7f1a58072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:41.549 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.565+0000 7f1a5e59c700 1 --2- 172.21.15.17:0/255640291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a58100e30 0x7f1a58072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:41.550 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.565+0000 7f1a5e59c700 1 -- 172.21.15.17:0/255640291 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1a580ffba0 msgr2=0x7f1a580742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:41.550 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.565+0000 7f1a5e59c700 1 --2- 172.21.15.17:0/255640291 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1a580ffba0 0x7f1a580742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:41.550 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.565+0000 7f1a5e59c700 1 -- 172.21.15.17:0/255640291 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a540095f0 con 0x7f1a581017e0 2023-07-14T12:01:41.550 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.565+0000 7f1a5d59a700 1 --2- 172.21.15.17:0/255640291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a58100e30 0x7f1a58072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T12:01:41.550 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.566+0000 7f1a5e59c700 1 --2- 172.21.15.17:0/255640291 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a581017e0 0x7f1a58072e40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1a5000ef30 tx=0x7f1a5000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:41.551 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.566+0000 7f1a4effd700 1 -- 172.21.15.17:0/255640291 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a50018070 con 0x7f1a581017e0 2023-07-14T12:01:41.551 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.566+0000 7f1a5ffff700 1 -- 172.21.15.17:0/255640291 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a5819fb70 con 0x7f1a581017e0 2023-07-14T12:01:41.551 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.566+0000 7f1a4effd700 1 -- 172.21.15.17:0/255640291 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a5000f040 con 0x7f1a581017e0 2023-07-14T12:01:41.551 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.566+0000 7f1a4effd700 1 -- 172.21.15.17:0/255640291 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5000cdb0 con 0x7f1a581017e0 2023-07-14T12:01:41.551 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.566+0000 7f1a5ffff700 1 -- 172.21.15.17:0/255640291 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a581a00d0 con 0x7f1a581017e0 2023-07-14T12:01:41.552 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.567+0000 7f1a5ffff700 1 -- 172.21.15.17:0/255640291 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a58106bc0 con 0x7f1a581017e0 2023-07-14T12:01:41.552 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.569+0000 7f1a4effd700 1 -- 172.21.15.17:0/255640291 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1a50003a00 con 0x7f1a581017e0 2023-07-14T12:01:41.553 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.570+0000 7f1a4effd700 1 --2- 172.21.15.17:0/255640291 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f1a440a1150 0x7f1a440a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:41.553 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.571+0000 7f1a4effd700 1 -- 172.21.15.17:0/255640291 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f1a500cc870 con 0x7f1a581017e0 2023-07-14T12:01:41.553 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.571+0000 7f1a5dd9b700 1 --2- 172.21.15.17:0/255640291 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f1a440a1150 0x7f1a440a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:41.553 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.571+0000 7f1a5dd9b700 1 --2- 172.21.15.17:0/255640291 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f1a440a1150 0x7f1a440a3600 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1a5400b2e0 tx=0x7f1a54005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:41.554 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.574+0000 7f1a4effd700 1 -- 172.21.15.17:0/255640291 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1a50095ca0 con 0x7f1a581017e0 2023-07-14T12:01:41.808 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.829+0000 7f1a5ffff700 1 -- 172.21.15.17:0/255640291 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a5802c850 con 0x7f1a581017e0 2023-07-14T12:01:41.809 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.829+0000 7f1a4effd700 1 -- 172.21.15.17:0/255640291 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10869 (secure 0 0 0) 0x7f1a500992c0 con 0x7f1a581017e0 2023-07-14T12:01:41.809 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T12:01:41.809 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/55384193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":55384193},{"type":"v1","addr":"172.21.15.17:6835","nonce":55384193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/424956554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":424956554},{"type":"v1","addr":"172.21.15.17:6837","nonce":424956554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T11:59:43.292866+0000","modified":"2023-07-14T12:00:17.489445+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24565,"mds_1":24557,"mds_2":24541,"mds_3":24533,"mds_4":24517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24452":{"gid":24452,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.93:6833/326319712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":326319712},{"type":"v1","addr":"172.21.15.93:6833","nonce":326319712}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24454":{"gid":24454,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6833/878913679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":878913679},{"type":"v1","addr":"172.21.15.97:6833","nonce":878913679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.93:6835/2779420503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":2779420503},{"type":"v1","addr":"172.21.15.93:6835","nonce":2779420503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/2740799393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2740799393},{"type":"v1","addr":"172.21.15.97:6835","nonce":2740799393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.93:6837/1880673911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":1880673911},{"type":"v1","addr":"172.21.15.93:6837","nonce":1880673911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.97:6837/3684187131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3684187131},{"type":"v1","addr":"172.21.15.97:6837","nonce":3684187131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"g","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.17:6839/2976493015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":2976493015},{"type":"v1","addr":"172.21.15.17:6839","nonce":2976493015}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.97:6839/3267815331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3267815331},{"type":"v1","addr":"172.21.15.97:6839","nonce":3267815331}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.93:6839/1281342428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":1281342428},{"type":"v1","addr":"172.21.15.93:6839","nonce":1281342428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6841/3905201143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6840","nonce":3905201143},{"type":"v1","addr":"172.21.15.17:6841","nonce":3905201143}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T12:01:41.812 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.832+0000 7f1a5ffff700 1 -- 172.21.15.17:0/255640291 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f1a440a1150 msgr2=0x7f1a440a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:41.812 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.832+0000 7f1a5ffff700 1 --2- 172.21.15.17:0/255640291 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f1a440a1150 0x7f1a440a3600 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1a5400b2e0 tx=0x7f1a54005fb0 comp rx=0 tx=0).stop 2023-07-14T12:01:41.812 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.832+0000 7f1a5ffff700 1 -- 172.21.15.17:0/255640291 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a581017e0 msgr2=0x7f1a58072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:41.813 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.832+0000 7f1a5ffff700 1 --2- 172.21.15.17:0/255640291 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a581017e0 0x7f1a58072e40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1a5000ef30 tx=0x7f1a5000c3a0 comp rx=0 tx=0).stop 2023-07-14T12:01:41.813 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.832+0000 7f1a5ffff700 1 -- 172.21.15.17:0/255640291 shutdown_connections 2023-07-14T12:01:41.813 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.832+0000 7f1a5ffff700 1 --2- 172.21.15.17:0/255640291 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f1a440a1150 0x7f1a440a3600 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:41.813 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.832+0000 7f1a5ffff700 1 --2- 172.21.15.17:0/255640291 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1a580ffba0 0x7f1a580742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:41.814 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.832+0000 7f1a5ffff700 1 --2- 172.21.15.17:0/255640291 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a581017e0 0x7f1a58072e40 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:41.814 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.832+0000 7f1a5ffff700 1 --2- 172.21.15.17:0/255640291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a58100e30 0x7f1a58072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:41.814 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.832+0000 7f1a5ffff700 1 -- 172.21.15.17:0/255640291 >> 172.21.15.17:0/255640291 conn(0x7f1a580fb1c0 msgr2=0x7f1a580fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:41.815 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.832+0000 7f1a5ffff700 1 -- 172.21.15.17:0/255640291 shutdown_connections 2023-07-14T12:01:41.815 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:41.832+0000 7f1a5ffff700 1 -- 172.21.15.17:0/255640291 wait complete. 2023-07-14T12:01:41.816 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 27 2023-07-14T12:01:41.830 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T12:01:42.035 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.056+0000 7f98d1dff700 1 -- 172.21.15.17:0/3824024300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98cc107a30 msgr2=0x7f98cc109e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:42.036 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.056+0000 7f98d1dff700 1 --2- 172.21.15.17:0/3824024300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98cc107a30 0x7f98cc109e50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f98c400b0a0 tx=0x7f98c400b3b0 comp rx=0 tx=0).stop 2023-07-14T12:01:42.036 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.056+0000 7f98d1dff700 1 -- 172.21.15.17:0/3824024300 shutdown_connections 2023-07-14T12:01:42.036 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.056+0000 7f98d1dff700 1 --2- 172.21.15.17:0/3824024300 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f98cc0744b0 0x7f98cc0729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:42.036 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.056+0000 7f98d1dff700 1 --2- 172.21.15.17:0/3824024300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98cc107a30 0x7f98cc109e50 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:42.036 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.056+0000 7f98d1dff700 1 --2- 172.21.15.17:0/3824024300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98cc072ee0 0x7f98cc073350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:42.037 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.056+0000 7f98d1dff700 1 -- 172.21.15.17:0/3824024300 >> 172.21.15.17:0/3824024300 conn(0x7f98cc0fb1d0 msgr2=0x7f98cc0fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:42.037 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.056+0000 7f98d1dff700 1 -- 172.21.15.17:0/3824024300 shutdown_connections 2023-07-14T12:01:42.037 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.056+0000 7f98d1dff700 1 -- 172.21.15.17:0/3824024300 wait complete. 2023-07-14T12:01:42.039 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.057+0000 7f98d1dff700 1 Processor -- start 2023-07-14T12:01:42.039 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.057+0000 7f98d1dff700 1 -- start start 2023-07-14T12:01:42.039 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.057+0000 7f98d1dff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98cc072ee0 0x7f98cc197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:42.039 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.058+0000 7f98d1dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98cc0744b0 0x7f98cc198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:42.040 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.058+0000 7f98d1dff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f98cc107a30 0x7f98cc19f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:42.040 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.058+0000 7f98d1dff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc19f790 con 0x7f98cc072ee0 2023-07-14T12:01:42.040 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.058+0000 7f98d1dff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc19f900 con 0x7f98cc107a30 2023-07-14T12:01:42.040 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.058+0000 7f98d1dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc19fac0 con 0x7f98cc0744b0 2023-07-14T12:01:42.041 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.058+0000 7f98cb7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98cc072ee0 0x7f98cc197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:42.041 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.058+0000 7f98cb7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98cc072ee0 0x7f98cc197c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44018/0 (socket says 172.21.15.17:44018) 2023-07-14T12:01:42.041 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.058+0000 7f98cb7fe700 1 -- 172.21.15.17:0/2707365091 learned_addr learned my addr 172.21.15.17:0/2707365091 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:42.041 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.058+0000 7f98caffd700 1 --2- 172.21.15.17:0/2707365091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98cc0744b0 0x7f98cc198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:42.041 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.058+0000 7f98cbfff700 1 --2- 172.21.15.17:0/2707365091 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f98cc107a30 0x7f98cc19f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:42.042 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.058+0000 7f98cb7fe700 1 -- 172.21.15.17:0/2707365091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98cc0744b0 msgr2=0x7f98cc198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:42.042 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.058+0000 7f98cb7fe700 1 --2- 172.21.15.17:0/2707365091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98cc0744b0 0x7f98cc198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:42.042 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.058+0000 7f98cb7fe700 1 -- 172.21.15.17:0/2707365091 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f98cc107a30 msgr2=0x7f98cc19f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:42.042 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.058+0000 7f98cb7fe700 1 --2- 172.21.15.17:0/2707365091 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f98cc107a30 0x7f98cc19f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:42.042 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.058+0000 7f98cb7fe700 1 -- 172.21.15.17:0/2707365091 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c4009d00 con 0x7f98cc072ee0 2023-07-14T12:01:42.042 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.058+0000 7f98cb7fe700 1 --2- 172.21.15.17:0/2707365091 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98cc072ee0 0x7f98cc197c50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f98bc00e970 tx=0x7f98bc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:42.043 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.059+0000 7f98c8ff9700 1 -- 172.21.15.17:0/2707365091 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98bc009790 con 0x7f98cc072ee0 2023-07-14T12:01:42.043 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.059+0000 7f98c8ff9700 1 -- 172.21.15.17:0/2707365091 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f98bc00c7e0 con 0x7f98cc072ee0 2023-07-14T12:01:42.043 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.059+0000 7f98d1dff700 1 -- 172.21.15.17:0/2707365091 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98cc19fda0 con 0x7f98cc072ee0 2023-07-14T12:01:42.043 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.059+0000 7f98c8ff9700 1 -- 172.21.15.17:0/2707365091 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98bc003c70 con 0x7f98cc072ee0 2023-07-14T12:01:42.043 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.059+0000 7f98d1dff700 1 -- 172.21.15.17:0/2707365091 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98cc1a01f0 con 0x7f98cc072ee0 2023-07-14T12:01:42.044 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.060+0000 7f98d1dff700 1 -- 172.21.15.17:0/2707365091 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98cc1061c0 con 0x7f98cc072ee0 2023-07-14T12:01:42.044 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.062+0000 7f98c8ff9700 1 -- 172.21.15.17:0/2707365091 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f98bc00cc50 con 0x7f98cc072ee0 2023-07-14T12:01:42.045 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.063+0000 7f98c8ff9700 1 --2- 172.21.15.17:0/2707365091 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f98b40a1150 0x7f98b40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:42.045 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.064+0000 7f98c8ff9700 1 -- 172.21.15.17:0/2707365091 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f98bc014070 con 0x7f98cc072ee0 2023-07-14T12:01:42.045 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.064+0000 7f98caffd700 1 --2- 172.21.15.17:0/2707365091 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f98b40a1150 0x7f98b40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:42.045 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.064+0000 7f98caffd700 1 --2- 172.21.15.17:0/2707365091 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f98b40a1150 0x7f98b40a3600 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f98c0005950 tx=0x7f98c000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:42.046 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.067+0000 7f98c8ff9700 1 -- 172.21.15.17:0/2707365091 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f98bc0961f0 con 0x7f98cc072ee0 2023-07-14T12:01:42.300 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.321+0000 7f98d1dff700 1 -- 172.21.15.17:0/2707365091 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f98cc02c850 con 0x7f98cc072ee0 2023-07-14T12:01:42.303 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.322+0000 7f98c8ff9700 1 -- 172.21.15.17:0/2707365091 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10869 (secure 0 0 0) 0x7f98bc003dd0 con 0x7f98cc072ee0 2023-07-14T12:01:42.304 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T12:01:42.304 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14472,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/55384193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":55384193},{"type":"v1","addr":"172.21.15.17:6835","nonce":55384193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/424956554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":424956554},{"type":"v1","addr":"172.21.15.17:6837","nonce":424956554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T11:59:43.292866+0000","modified":"2023-07-14T12:00:17.489445+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24565,"mds_1":24557,"mds_2":24541,"mds_3":24533,"mds_4":24517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24452":{"gid":24452,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.93:6833/326319712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":326319712},{"type":"v1","addr":"172.21.15.93:6833","nonce":326319712}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24454":{"gid":24454,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6833/878913679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":878913679},{"type":"v1","addr":"172.21.15.97:6833","nonce":878913679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.93:6835/2779420503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":2779420503},{"type":"v1","addr":"172.21.15.93:6835","nonce":2779420503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/2740799393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2740799393},{"type":"v1","addr":"172.21.15.97:6835","nonce":2740799393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.93:6837/1880673911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":1880673911},{"type":"v1","addr":"172.21.15.93:6837","nonce":1880673911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.97:6837/3684187131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3684187131},{"type":"v1","addr":"172.21.15.97:6837","nonce":3684187131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"g","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.17:6839/2976493015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":2976493015},{"type":"v1","addr":"172.21.15.17:6839","nonce":2976493015}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.97:6839/3267815331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3267815331},{"type":"v1","addr":"172.21.15.97:6839","nonce":3267815331}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.93:6839/1281342428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":1281342428},{"type":"v1","addr":"172.21.15.93:6839","nonce":1281342428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6841/3905201143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6840","nonce":3905201143},{"type":"v1","addr":"172.21.15.17:6841","nonce":3905201143}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T12:01:42.305 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.325+0000 7f98d1dff700 1 -- 172.21.15.17:0/2707365091 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f98b40a1150 msgr2=0x7f98b40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:42.305 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.325+0000 7f98d1dff700 1 --2- 172.21.15.17:0/2707365091 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f98b40a1150 0x7f98b40a3600 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f98c0005950 tx=0x7f98c000b410 comp rx=0 tx=0).stop 2023-07-14T12:01:42.305 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.325+0000 7f98d1dff700 1 -- 172.21.15.17:0/2707365091 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98cc072ee0 msgr2=0x7f98cc197c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:42.306 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.325+0000 7f98d1dff700 1 --2- 172.21.15.17:0/2707365091 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98cc072ee0 0x7f98cc197c50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f98bc00e970 tx=0x7f98bc00ec80 comp rx=0 tx=0).stop 2023-07-14T12:01:42.306 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.325+0000 7f98d1dff700 1 -- 172.21.15.17:0/2707365091 shutdown_connections 2023-07-14T12:01:42.306 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.325+0000 7f98d1dff700 1 --2- 172.21.15.17:0/2707365091 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f98b40a1150 0x7f98b40a3600 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:42.306 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.325+0000 7f98d1dff700 1 --2- 172.21.15.17:0/2707365091 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f98cc107a30 0x7f98cc19f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:42.306 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.325+0000 7f98d1dff700 1 --2- 172.21.15.17:0/2707365091 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98cc072ee0 0x7f98cc197c50 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:42.307 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.325+0000 7f98d1dff700 1 --2- 172.21.15.17:0/2707365091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98cc0744b0 0x7f98cc198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:42.307 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.325+0000 7f98d1dff700 1 -- 172.21.15.17:0/2707365091 >> 172.21.15.17:0/2707365091 conn(0x7f98cc0fb1d0 msgr2=0x7f98cc0fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:42.307 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.325+0000 7f98d1dff700 1 -- 172.21.15.17:0/2707365091 shutdown_connections 2023-07-14T12:01:42.307 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.325+0000 7f98d1dff700 1 -- 172.21.15.17:0/2707365091 wait complete. 2023-07-14T12:01:42.307 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 27 2023-07-14T12:01:42.323 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T12:01:42.527 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.548+0000 7ff4dffff700 1 -- 172.21.15.17:0/2713914463 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4d8102f40 msgr2=0x7ff4d8105320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:42.528 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.548+0000 7ff4dffff700 1 --2- 172.21.15.17:0/2713914463 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4d8102f40 0x7ff4d8105320 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8009990 tx=0x7ff4c8009ca0 comp rx=0 tx=0).stop 2023-07-14T12:01:42.528 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.548+0000 7ff4dffff700 1 -- 172.21.15.17:0/2713914463 shutdown_connections 2023-07-14T12:01:42.528 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.548+0000 7ff4dffff700 1 --2- 172.21.15.17:0/2713914463 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff4d8100620 0x7ff4d8102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:42.528 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.548+0000 7ff4dffff700 1 --2- 172.21.15.17:0/2713914463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4d8105860 0x7ff4d8107cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:42.529 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.548+0000 7ff4dffff700 1 --2- 172.21.15.17:0/2713914463 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4d8102f40 0x7ff4d8105320 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:42.529 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.548+0000 7ff4dffff700 1 -- 172.21.15.17:0/2713914463 >> 172.21.15.17:0/2713914463 conn(0x7ff4d80fa120 msgr2=0x7ff4d80fc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:42.529 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.548+0000 7ff4dffff700 1 -- 172.21.15.17:0/2713914463 shutdown_connections 2023-07-14T12:01:42.529 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.548+0000 7ff4dffff700 1 -- 172.21.15.17:0/2713914463 wait complete. 2023-07-14T12:01:42.531 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.549+0000 7ff4dffff700 1 Processor -- start 2023-07-14T12:01:42.531 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.549+0000 7ff4dffff700 1 -- start start 2023-07-14T12:01:42.531 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.550+0000 7ff4dffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff4d8100620 0x7ff4d81958c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:42.532 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.550+0000 7ff4dffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4d8102f40 0x7ff4d8195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:42.532 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.550+0000 7ff4dffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4d8105860 0x7ff4d819ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:42.532 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.550+0000 7ff4dffff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d819d440 con 0x7ff4d8105860 2023-07-14T12:01:42.532 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.550+0000 7ff4dffff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d819d580 con 0x7ff4d8100620 2023-07-14T12:01:42.532 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.550+0000 7ff4dffff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d819d710 con 0x7ff4d8102f40 2023-07-14T12:01:42.533 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.550+0000 7ff4ddd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff4d8100620 0x7ff4d81958c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:42.533 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.550+0000 7ff4de59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4d8105860 0x7ff4d819ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:42.533 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.550+0000 7ff4ddd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff4d8100620 0x7ff4d81958c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.17:38188/0 (socket says 172.21.15.17:38188) 2023-07-14T12:01:42.533 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.550+0000 7ff4ddd9b700 1 -- 172.21.15.17:0/4156751660 learned_addr learned my addr 172.21.15.17:0/4156751660 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:42.533 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.550+0000 7ff4dd59a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4d8102f40 0x7ff4d8195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:42.534 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.550+0000 7ff4de59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4d8105860 0x7ff4d819ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44026/0 (socket says 172.21.15.17:44026) 2023-07-14T12:01:42.534 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.551+0000 7ff4ddd9b700 1 -- 172.21.15.17:0/4156751660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4d8102f40 msgr2=0x7ff4d8195e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:42.534 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.551+0000 7ff4ddd9b700 1 --2- 172.21.15.17:0/4156751660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4d8102f40 0x7ff4d8195e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:42.534 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.551+0000 7ff4ddd9b700 1 -- 172.21.15.17:0/4156751660 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4d8105860 msgr2=0x7ff4d819ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:42.534 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.551+0000 7ff4ddd9b700 1 --2- 172.21.15.17:0/4156751660 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4d8105860 0x7ff4d819ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:42.535 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.551+0000 7ff4ddd9b700 1 -- 172.21.15.17:0/4156751660 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c80095f0 con 0x7ff4d8100620 2023-07-14T12:01:42.535 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.551+0000 7ff4de59c700 1 --2- 172.21.15.17:0/4156751660 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4d8105860 0x7ff4d819ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T12:01:42.535 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.551+0000 7ff4ddd9b700 1 --2- 172.21.15.17:0/4156751660 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff4d8100620 0x7ff4d81958c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff4d400e9a0 tx=0x7ff4d400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:42.535 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.551+0000 7ff4ceffd700 1 -- 172.21.15.17:0/4156751660 <== mon.2 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d4009790 con 0x7ff4d8100620 2023-07-14T12:01:42.535 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.551+0000 7ff4dffff700 1 -- 172.21.15.17:0/4156751660 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4d80fe530 con 0x7ff4d8100620 2023-07-14T12:01:42.535 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.551+0000 7ff4ceffd700 1 -- 172.21.15.17:0/4156751660 <== mon.2 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4d4010430 con 0x7ff4d8100620 2023-07-14T12:01:42.536 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.551+0000 7ff4ceffd700 1 -- 172.21.15.17:0/4156751660 <== mon.2 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d4018680 con 0x7ff4d8100620 2023-07-14T12:01:42.536 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.551+0000 7ff4dffff700 1 -- 172.21.15.17:0/4156751660 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4d80fea40 con 0x7ff4d8100620 2023-07-14T12:01:42.536 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.552+0000 7ff4dffff700 1 -- 172.21.15.17:0/4156751660 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4d80fcca0 con 0x7ff4d8100620 2023-07-14T12:01:42.537 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.555+0000 7ff4ceffd700 1 -- 172.21.15.17:0/4156751660 <== mon.2 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff4d40105a0 con 0x7ff4d8100620 2023-07-14T12:01:42.537 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.556+0000 7ff4ceffd700 1 --2- 172.21.15.17:0/4156751660 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7ff4c40a1150 0x7ff4c40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:42.537 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.556+0000 7ff4ceffd700 1 -- 172.21.15.17:0/4156751660 <== mon.2 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7ff4d4014070 con 0x7ff4d8100620 2023-07-14T12:01:42.538 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.556+0000 7ff4dd59a700 1 --2- 172.21.15.17:0/4156751660 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7ff4c40a1150 0x7ff4c40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:42.538 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.557+0000 7ff4dd59a700 1 --2- 172.21.15.17:0/4156751660 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7ff4c40a1150 0x7ff4c40a3600 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff4c800b2e0 tx=0x7ff4c8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:42.538 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.558+0000 7ff4ceffd700 1 -- 172.21.15.17:0/4156751660 <== mon.2 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff4d4096410 con 0x7ff4d8100620 2023-07-14T12:01:42.745 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.767+0000 7ff4dffff700 1 -- 172.21.15.17:0/4156751660 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff4d802c850 con 0x7ff4d8100620 2023-07-14T12:01:42.747 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.768+0000 7ff4ceffd700 1 -- 172.21.15.17:0/4156751660 <== mon.2 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24486 (secure 0 0 0) 0x7ff4d4026090 con 0x7ff4d8100620 2023-07-14T12:01:42.748 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-07-14T12:01:42.748 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":92,"fsid":"b3c386cc-223c-11ee-9b33-001a4aab830c","created":"2023-07-14T11:51:59.992960+0000","modified":"2023-07-14T12:01:07.806062+0000","last_up_change":"2023-07-14T11:58:55.037769+0000","last_in_change":"2023-07-14T11:58:37.114870+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T11:55:28.237002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T11:59:41.560502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T11:59:42.725461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-14T11:59:46.267029+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"57003438-a83c-4c8e-a029-56414740895c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6803","nonce":3588366384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6805","nonce":3588366384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6809","nonce":3588366384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":3588366384},{"type":"v1","addr":"172.21.15.17:6807","nonce":3588366384}]},"public_addr":"172.21.15.17:6803/3588366384","cluster_addr":"172.21.15.17:6805/3588366384","heartbeat_back_addr":"172.21.15.17:6809/3588366384","heartbeat_front_addr":"172.21.15.17:6807/3588366384","state":["exists","up"]},{"osd":1,"uuid":"f56b6337-d8a9-4411-b9d8-b6863dc98556","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6801","nonce":3147203064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6803","nonce":3147203064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6807","nonce":3147203064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":3147203064},{"type":"v1","addr":"172.21.15.93:6805","nonce":3147203064}]},"public_addr":"172.21.15.93:6801/3147203064","cluster_addr":"172.21.15.93:6803/3147203064","heartbeat_back_addr":"172.21.15.93:6807/3147203064","heartbeat_front_addr":"172.21.15.93:6805/3147203064","state":["exists","up"]},{"osd":2,"uuid":"fc34bb7f-551f-46e1-99c0-59924d6c1346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6801","nonce":1877269815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6803","nonce":1877269815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6807","nonce":1877269815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1877269815},{"type":"v1","addr":"172.21.15.97:6805","nonce":1877269815}]},"public_addr":"172.21.15.97:6801/1877269815","cluster_addr":"172.21.15.97:6803/1877269815","heartbeat_back_addr":"172.21.15.97:6807/1877269815","heartbeat_front_addr":"172.21.15.97:6805/1877269815","state":["exists","up"]},{"osd":3,"uuid":"37a1583c-4fb8-412a-a0c8-f83b1ff26a91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6811","nonce":2784619821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6813","nonce":2784619821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6817","nonce":2784619821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":2784619821},{"type":"v1","addr":"172.21.15.17:6815","nonce":2784619821}]},"public_addr":"172.21.15.17:6811/2784619821","cluster_addr":"172.21.15.17:6813/2784619821","heartbeat_back_addr":"172.21.15.17:6817/2784619821","heartbeat_front_addr":"172.21.15.17:6815/2784619821","state":["exists","up"]},{"osd":4,"uuid":"f4b9f377-d519-4b14-ac4a-f1debaeed9f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6809","nonce":3151801173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6811","nonce":3151801173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6815","nonce":3151801173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":3151801173},{"type":"v1","addr":"172.21.15.93:6813","nonce":3151801173}]},"public_addr":"172.21.15.93:6809/3151801173","cluster_addr":"172.21.15.93:6811/3151801173","heartbeat_back_addr":"172.21.15.93:6815/3151801173","heartbeat_front_addr":"172.21.15.93:6813/3151801173","state":["exists","up"]},{"osd":5,"uuid":"2d2af354-0535-4ef9-b166-3ef219498db8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6809","nonce":2184413924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6811","nonce":2184413924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6815","nonce":2184413924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2184413924},{"type":"v1","addr":"172.21.15.97:6813","nonce":2184413924}]},"public_addr":"172.21.15.97:6809/2184413924","cluster_addr":"172.21.15.97:6811/2184413924","heartbeat_back_addr":"172.21.15.97:6815/2184413924","heartbeat_front_addr":"172.21.15.97:6813/2184413924","state":["exists","up"]},{"osd":6,"uuid":"9cdf0c4b-62dc-4168-864e-a9d1650a9ea8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6819","nonce":50486581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6821","nonce":50486581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6825","nonce":50486581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":50486581},{"type":"v1","addr":"172.21.15.17:6823","nonce":50486581}]},"public_addr":"172.21.15.17:6819/50486581","cluster_addr":"172.21.15.17:6821/50486581","heartbeat_back_addr":"172.21.15.17:6825/50486581","heartbeat_front_addr":"172.21.15.17:6823/50486581","state":["exists","up"]},{"osd":7,"uuid":"09f60823-5491-45cd-9324-6ae0d56ce262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6817","nonce":2596912392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6819","nonce":2596912392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6823","nonce":2596912392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":2596912392},{"type":"v1","addr":"172.21.15.93:6821","nonce":2596912392}]},"public_addr":"172.21.15.93:6817/2596912392","cluster_addr":"172.21.15.93:6819/2596912392","heartbeat_back_addr":"172.21.15.93:6823/2596912392","heartbeat_front_addr":"172.21.15.93:6821/2596912392","state":["exists","up"]},{"osd":8,"uuid":"8c5771c9-718e-4313-a2b6-5e1c37c53f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6817","nonce":1965492859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6819","nonce":1965492859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6823","nonce":1965492859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1965492859},{"type":"v1","addr":"172.21.15.97:6821","nonce":1965492859}]},"public_addr":"172.21.15.97:6817/1965492859","cluster_addr":"172.21.15.97:6819/1965492859","heartbeat_back_addr":"172.21.15.97:6823/1965492859","heartbeat_front_addr":"172.21.15.97:6821/1965492859","state":["exists","up"]},{"osd":9,"uuid":"1197b8a3-c09f-4c57-aba1-9e988a1620b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6827","nonce":2512645700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6829","nonce":2512645700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6833","nonce":2512645700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2512645700},{"type":"v1","addr":"172.21.15.17:6831","nonce":2512645700}]},"public_addr":"172.21.15.17:6827/2512645700","cluster_addr":"172.21.15.17:6829/2512645700","heartbeat_back_addr":"172.21.15.17:6833/2512645700","heartbeat_front_addr":"172.21.15.17:6831/2512645700","state":["exists","up"]},{"osd":10,"uuid":"f99918c2-cdd8-4a7a-be2e-d1d6e0f828a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6825","nonce":264429558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6827","nonce":264429558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6831","nonce":264429558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":264429558},{"type":"v1","addr":"172.21.15.93:6829","nonce":264429558}]},"public_addr":"172.21.15.93:6825/264429558","cluster_addr":"172.21.15.93:6827/264429558","heartbeat_back_addr":"172.21.15.93:6831/264429558","heartbeat_front_addr":"172.21.15.93:6829/264429558","state":["exists","up"]},{"osd":11,"uuid":"6bbb5b79-8d4d-441f-9583-203df6c78cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6825","nonce":3389913754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6827","nonce":3389913754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6831","nonce":3389913754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3389913754},{"type":"v1","addr":"172.21.15.97:6829","nonce":3389913754}]},"public_addr":"172.21.15.97:6825/3389913754","cluster_addr":"172.21.15.97:6827/3389913754","heartbeat_back_addr":"172.21.15.97:6831/3389913754","heartbeat_front_addr":"172.21.15.97:6829/3389913754","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:54:41.079618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:03.207899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:24.136961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:55:50.392609+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:56:12.136237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:56:33.212314+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:00.243646+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:22.059234+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:57:43.232399+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:10.043896+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:31.910780+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T11:58:53.158983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.9","mappings":[{"from":5,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:0/2084033276":"2023-07-15T11:53:05.639597+0000","172.21.15.17:6801/1059284716":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/3651785636":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/1373538024":"2023-07-15T11:52:37.887946+0000","172.21.15.17:0/3560395020":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6800/1059284716":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/4262912861":"2023-07-15T11:53:05.639597+0000","172.21.15.17:0/2258375202":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6800/2804227742":"2023-07-15T11:52:37.887946+0000","172.21.15.17:6801/2804227742":"2023-07-15T11:52:37.887946+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T12:01:42.750 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.771+0000 7ff4dffff700 1 -- 172.21.15.17:0/4156751660 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7ff4c40a1150 msgr2=0x7ff4c40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:42.751 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.771+0000 7ff4dffff700 1 --2- 172.21.15.17:0/4156751660 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7ff4c40a1150 0x7ff4c40a3600 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff4c800b2e0 tx=0x7ff4c8005fb0 comp rx=0 tx=0).stop 2023-07-14T12:01:42.751 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.771+0000 7ff4dffff700 1 -- 172.21.15.17:0/4156751660 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff4d8100620 msgr2=0x7ff4d81958c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:42.751 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.771+0000 7ff4dffff700 1 --2- 172.21.15.17:0/4156751660 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff4d8100620 0x7ff4d81958c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff4d400e9a0 tx=0x7ff4d400ecb0 comp rx=0 tx=0).stop 2023-07-14T12:01:42.751 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.771+0000 7ff4dffff700 1 -- 172.21.15.17:0/4156751660 shutdown_connections 2023-07-14T12:01:42.751 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.771+0000 7ff4dffff700 1 --2- 172.21.15.17:0/4156751660 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7ff4c40a1150 0x7ff4c40a3600 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:42.752 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.771+0000 7ff4dffff700 1 --2- 172.21.15.17:0/4156751660 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff4d8100620 0x7ff4d81958c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:42.752 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.771+0000 7ff4dffff700 1 --2- 172.21.15.17:0/4156751660 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4d8105860 0x7ff4d819ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:42.752 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.771+0000 7ff4dffff700 1 --2- 172.21.15.17:0/4156751660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4d8102f40 0x7ff4d8195e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:42.752 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.771+0000 7ff4dffff700 1 -- 172.21.15.17:0/4156751660 >> 172.21.15.17:0/4156751660 conn(0x7ff4d80fa120 msgr2=0x7ff4d8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:42.752 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.772+0000 7ff4dffff700 1 -- 172.21.15.17:0/4156751660 shutdown_connections 2023-07-14T12:01:42.753 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.772+0000 7ff4dffff700 1 -- 172.21.15.17:0/4156751660 wait complete. 2023-07-14T12:01:42.770 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-07-14T12:01:42.977 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.998+0000 7fc9febff700 1 -- 172.21.15.17:0/1270019650 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9f8101760 msgr2=0x7fc9f8108020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:42.978 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.998+0000 7fc9febff700 1 --2- 172.21.15.17:0/1270019650 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9f8101760 0x7fc9f8108020 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc9f000b0a0 tx=0x7fc9f000b3b0 comp rx=0 tx=0).stop 2023-07-14T12:01:42.978 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.998+0000 7fc9febff700 1 -- 172.21.15.17:0/1270019650 shutdown_connections 2023-07-14T12:01:42.978 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.998+0000 7fc9febff700 1 --2- 172.21.15.17:0/1270019650 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9f8101760 0x7fc9f8108020 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:42.978 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.998+0000 7fc9febff700 1 --2- 172.21.15.17:0/1270019650 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9f80ffbe0 0x7fc9f80ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:42.978 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.998+0000 7fc9febff700 1 --2- 172.21.15.17:0/1270019650 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc9f8100dd0 0x7fc9f8101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:42.979 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.998+0000 7fc9febff700 1 -- 172.21.15.17:0/1270019650 >> 172.21.15.17:0/1270019650 conn(0x7fc9f80fb200 msgr2=0x7fc9f80fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:42.979 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.998+0000 7fc9febff700 1 -- 172.21.15.17:0/1270019650 shutdown_connections 2023-07-14T12:01:42.979 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.998+0000 7fc9febff700 1 -- 172.21.15.17:0/1270019650 wait complete. 2023-07-14T12:01:42.981 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.999+0000 7fc9febff700 1 Processor -- start 2023-07-14T12:01:42.981 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:42.999+0000 7fc9febff700 1 -- start start 2023-07-14T12:01:42.981 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.000+0000 7fc9febff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc9f80ffbe0 0x7fc9f80742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:42.981 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.000+0000 7fc9febff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9f8100dd0 0x7fc9f8072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:42.982 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.000+0000 7fc9febff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9f8101760 0x7fc9f8072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:42.982 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.000+0000 7fc9febff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f80735e0 con 0x7fc9f8100dd0 2023-07-14T12:01:42.982 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.000+0000 7fc9febff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f8073750 con 0x7fc9f80ffbe0 2023-07-14T12:01:42.982 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.000+0000 7fc9febff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f819fa90 con 0x7fc9f8101760 2023-07-14T12:01:42.982 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.000+0000 7fc9effff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9f8100dd0 0x7fc9f8072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:42.983 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.000+0000 7fc9effff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9f8100dd0 0x7fc9f8072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52250/0 (socket says 172.21.15.17:52250) 2023-07-14T12:01:42.983 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.000+0000 7fc9effff700 1 -- 172.21.15.17:0/3846934238 learned_addr learned my addr 172.21.15.17:0/3846934238 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:42.983 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.000+0000 7fc9fc99b700 1 --2- 172.21.15.17:0/3846934238 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc9f80ffbe0 0x7fc9f80742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:42.983 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.000+0000 7fc9fd19c700 1 --2- 172.21.15.17:0/3846934238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9f8101760 0x7fc9f8072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:42.983 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.001+0000 7fc9effff700 1 -- 172.21.15.17:0/3846934238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9f8101760 msgr2=0x7fc9f8072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:42.984 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.001+0000 7fc9effff700 1 --2- 172.21.15.17:0/3846934238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9f8101760 0x7fc9f8072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:42.984 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.001+0000 7fc9effff700 1 -- 172.21.15.17:0/3846934238 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc9f80ffbe0 msgr2=0x7fc9f80742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:42.984 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.001+0000 7fc9effff700 1 --2- 172.21.15.17:0/3846934238 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc9f80ffbe0 0x7fc9f80742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:42.984 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.001+0000 7fc9effff700 1 -- 172.21.15.17:0/3846934238 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e8009520 con 0x7fc9f8100dd0 2023-07-14T12:01:42.984 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.001+0000 7fc9fc99b700 1 --2- 172.21.15.17:0/3846934238 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc9f80ffbe0 0x7fc9f80742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T12:01:42.985 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.001+0000 7fc9effff700 1 --2- 172.21.15.17:0/3846934238 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9f8100dd0 0x7fc9f8072900 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc9e800eb30 tx=0x7fc9e800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:42.985 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.001+0000 7fc9edffb700 1 -- 172.21.15.17:0/3846934238 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e8009880 con 0x7fc9f8100dd0 2023-07-14T12:01:42.985 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.001+0000 7fc9febff700 1 -- 172.21.15.17:0/3846934238 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9f0009d00 con 0x7fc9f8100dd0 2023-07-14T12:01:42.985 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.001+0000 7fc9edffb700 1 -- 172.21.15.17:0/3846934238 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9e80044e0 con 0x7fc9f8100dd0 2023-07-14T12:01:42.985 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.001+0000 7fc9edffb700 1 -- 172.21.15.17:0/3846934238 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e8018430 con 0x7fc9f8100dd0 2023-07-14T12:01:42.985 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.001+0000 7fc9febff700 1 -- 172.21.15.17:0/3846934238 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9f81a00f0 con 0x7fc9f8100dd0 2023-07-14T12:01:42.986 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.002+0000 7fc9febff700 1 -- 172.21.15.17:0/3846934238 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9f8107080 con 0x7fc9f8100dd0 2023-07-14T12:01:42.987 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.004+0000 7fc9edffb700 1 -- 172.21.15.17:0/3846934238 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc9e8003ec0 con 0x7fc9f8100dd0 2023-07-14T12:01:42.987 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.006+0000 7fc9edffb700 1 --2- 172.21.15.17:0/3846934238 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fc9e00a1150 0x7fc9e00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:42.987 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.006+0000 7fc9edffb700 1 -- 172.21.15.17:0/3846934238 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fc9e8014070 con 0x7fc9f8100dd0 2023-07-14T12:01:42.987 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.006+0000 7fc9fc99b700 1 --2- 172.21.15.17:0/3846934238 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fc9e00a1150 0x7fc9e00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:42.987 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.007+0000 7fc9fc99b700 1 --2- 172.21.15.17:0/3846934238 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fc9e00a1150 0x7fc9e00a3600 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8100c30 tx=0x7fc9f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:42.988 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.009+0000 7fc9edffb700 1 -- 172.21.15.17:0/3846934238 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc9e80964f0 con 0x7fc9f8100dd0 2023-07-14T12:01:43.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:42 smithi093 ceph-mon[128922]: pgmap v340: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-07-14T12:01:43.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:42 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/255640291' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T12:01:43.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:42 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/2707365091' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T12:01:43.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:42 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/4156751660' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T12:01:43.133 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:42 smithi097 ceph-mon[129313]: pgmap v340: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-07-14T12:01:43.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:42 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/255640291' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T12:01:43.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:42 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/2707365091' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T12:01:43.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:42 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/4156751660' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T12:01:43.143 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:42 smithi017 ceph-mon[126339]: pgmap v340: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-07-14T12:01:43.143 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:42 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/255640291' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T12:01:43.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:42 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/2707365091' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T12:01:43.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:42 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/4156751660' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T12:01:43.239 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.261+0000 7fc9febff700 1 -- 172.21.15.17:0/3846934238 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc9f802c850 con 0x7fc9f8100dd0 2023-07-14T12:01:43.246 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.267+0000 7fc9edffb700 1 -- 172.21.15.17:0/3846934238 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc9e8099b10 con 0x7fc9f8100dd0 2023-07-14T12:01:43.250 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.271+0000 7fc9febff700 1 -- 172.21.15.17:0/3846934238 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fc9e00a1150 msgr2=0x7fc9e00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:43.251 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.271+0000 7fc9febff700 1 --2- 172.21.15.17:0/3846934238 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fc9e00a1150 0x7fc9e00a3600 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8100c30 tx=0x7fc9f4009260 comp rx=0 tx=0).stop 2023-07-14T12:01:43.251 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.271+0000 7fc9febff700 1 -- 172.21.15.17:0/3846934238 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9f8100dd0 msgr2=0x7fc9f8072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:43.251 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.271+0000 7fc9febff700 1 --2- 172.21.15.17:0/3846934238 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9f8100dd0 0x7fc9f8072900 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc9e800eb30 tx=0x7fc9e800eef0 comp rx=0 tx=0).stop 2023-07-14T12:01:43.251 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.271+0000 7fc9febff700 1 -- 172.21.15.17:0/3846934238 shutdown_connections 2023-07-14T12:01:43.251 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.271+0000 7fc9febff700 1 --2- 172.21.15.17:0/3846934238 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fc9e00a1150 0x7fc9e00a3600 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:43.252 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.271+0000 7fc9febff700 1 --2- 172.21.15.17:0/3846934238 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc9f80ffbe0 0x7fc9f80742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:43.252 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.271+0000 7fc9febff700 1 --2- 172.21.15.17:0/3846934238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9f8101760 0x7fc9f8072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:43.252 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.271+0000 7fc9febff700 1 --2- 172.21.15.17:0/3846934238 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9f8100dd0 0x7fc9f8072900 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:43.252 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.271+0000 7fc9febff700 1 -- 172.21.15.17:0/3846934238 >> 172.21.15.17:0/3846934238 conn(0x7fc9f80fb200 msgr2=0x7fc9f81041c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:43.252 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.271+0000 7fc9febff700 1 -- 172.21.15.17:0/3846934238 shutdown_connections 2023-07-14T12:01:43.253 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.272+0000 7fc9febff700 1 -- 172.21.15.17:0/3846934238 wait complete. 2023-07-14T12:01:43.253 INFO:teuthology.orchestra.run.smithi017.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-07-14T12:01:43.276 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-07-14T12:01:43.496 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.517+0000 7f2c4f3ff700 1 -- 172.21.15.17:0/3629540791 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c48100dd0 msgr2=0x7f2c48101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:43.497 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.517+0000 7f2c4f3ff700 1 --2- 172.21.15.17:0/3629540791 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c48100dd0 0x7f2c48101220 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2c38009990 tx=0x7f2c38009ca0 comp rx=0 tx=0).stop 2023-07-14T12:01:43.497 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.517+0000 7f2c4f3ff700 1 -- 172.21.15.17:0/3629540791 shutdown_connections 2023-07-14T12:01:43.497 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.517+0000 7f2c4f3ff700 1 --2- 172.21.15.17:0/3629540791 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c48101760 0x7f2c48108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:43.498 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.517+0000 7f2c4f3ff700 1 --2- 172.21.15.17:0/3629540791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c480ffbe0 0x7f2c480ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:43.498 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.517+0000 7f2c4f3ff700 1 --2- 172.21.15.17:0/3629540791 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c48100dd0 0x7f2c48101220 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:43.498 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.517+0000 7f2c4f3ff700 1 -- 172.21.15.17:0/3629540791 >> 172.21.15.17:0/3629540791 conn(0x7f2c480fb200 msgr2=0x7f2c480fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:43.498 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.517+0000 7f2c4f3ff700 1 -- 172.21.15.17:0/3629540791 shutdown_connections 2023-07-14T12:01:43.499 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.517+0000 7f2c4f3ff700 1 -- 172.21.15.17:0/3629540791 wait complete. 2023-07-14T12:01:43.500 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.518+0000 7f2c4f3ff700 1 Processor -- start 2023-07-14T12:01:43.501 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.518+0000 7f2c4f3ff700 1 -- start start 2023-07-14T12:01:43.501 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.519+0000 7f2c4f3ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c480ffbe0 0x7f2c480742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:43.501 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.519+0000 7f2c4f3ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c48100dd0 0x7f2c48072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:43.501 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.519+0000 7f2c4f3ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c48101760 0x7f2c48072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:43.502 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.519+0000 7f2c4f3ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c480735e0 con 0x7f2c48101760 2023-07-14T12:01:43.502 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.519+0000 7f2c4f3ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c48073750 con 0x7f2c48100dd0 2023-07-14T12:01:43.502 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.519+0000 7f2c4f3ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4819f990 con 0x7f2c480ffbe0 2023-07-14T12:01:43.502 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.519+0000 7f2c4d19b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c480ffbe0 0x7f2c480742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:43.502 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.519+0000 7f2c4c99a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c48100dd0 0x7f2c48072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:43.503 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.519+0000 7f2c4d99c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c48101760 0x7f2c48072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:43.503 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.519+0000 7f2c4c99a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c48100dd0 0x7f2c48072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.17:59666/0 (socket says 172.21.15.17:59666) 2023-07-14T12:01:43.503 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.519+0000 7f2c4c99a700 1 -- 172.21.15.17:0/1008224092 learned_addr learned my addr 172.21.15.17:0/1008224092 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:43.503 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.519+0000 7f2c4c99a700 1 -- 172.21.15.17:0/1008224092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c480ffbe0 msgr2=0x7f2c480742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:43.504 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.519+0000 7f2c4c99a700 1 --2- 172.21.15.17:0/1008224092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c480ffbe0 0x7f2c480742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:43.504 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.519+0000 7f2c4c99a700 1 -- 172.21.15.17:0/1008224092 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c48101760 msgr2=0x7f2c48072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:43.504 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.519+0000 7f2c4c99a700 1 --2- 172.21.15.17:0/1008224092 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c48101760 0x7f2c48072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:43.504 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.519+0000 7f2c4c99a700 1 -- 172.21.15.17:0/1008224092 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c380095f0 con 0x7f2c48100dd0 2023-07-14T12:01:43.505 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.520+0000 7f2c4c99a700 1 --2- 172.21.15.17:0/1008224092 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c48100dd0 0x7f2c48072900 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2c3800c010 tx=0x7f2c38004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:43.505 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.520+0000 7f2c3e7fc700 1 -- 172.21.15.17:0/1008224092 <== mon.2 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3801d070 con 0x7f2c48100dd0 2023-07-14T12:01:43.505 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.520+0000 7f2c4f3ff700 1 -- 172.21.15.17:0/1008224092 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c4819fc10 con 0x7f2c48100dd0 2023-07-14T12:01:43.505 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.520+0000 7f2c4f3ff700 1 -- 172.21.15.17:0/1008224092 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c481a0030 con 0x7f2c48100dd0 2023-07-14T12:01:43.505 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.520+0000 7f2c3e7fc700 1 -- 172.21.15.17:0/1008224092 <== mon.2 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c38005030 con 0x7f2c48100dd0 2023-07-14T12:01:43.506 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.520+0000 7f2c3e7fc700 1 -- 172.21.15.17:0/1008224092 <== mon.2 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c38004bc0 con 0x7f2c48100dd0 2023-07-14T12:01:43.506 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.521+0000 7f2c4f3ff700 1 -- 172.21.15.17:0/1008224092 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c4810b400 con 0x7f2c48100dd0 2023-07-14T12:01:43.507 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.523+0000 7f2c3e7fc700 1 -- 172.21.15.17:0/1008224092 <== mon.2 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2c380051a0 con 0x7f2c48100dd0 2023-07-14T12:01:43.507 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.524+0000 7f2c3e7fc700 1 --2- 172.21.15.17:0/1008224092 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f2c340a11d0 0x7f2c340a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:43.507 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.524+0000 7f2c3e7fc700 1 -- 172.21.15.17:0/1008224092 <== mon.2 v2:172.21.15.93:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9508+0+0 (secure 0 0 0) 0x7f2c380ce090 con 0x7f2c48100dd0 2023-07-14T12:01:43.508 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.524+0000 7f2c4d19b700 1 --2- 172.21.15.17:0/1008224092 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f2c340a11d0 0x7f2c340a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:43.508 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.525+0000 7f2c4d19b700 1 --2- 172.21.15.17:0/1008224092 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f2c340a11d0 0x7f2c340a3680 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2c48100c30 tx=0x7f2c44009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:43.508 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.528+0000 7f2c3e7fc700 1 -- 172.21.15.17:0/1008224092 <== mon.2 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2c3805a790 con 0x7f2c48100dd0 2023-07-14T12:01:43.610 INFO:journalctl@ceph.mds.e.smithi093.stdout:Jul 14 12:01:43 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-e[153790]: ignoring --setuser ceph since I am not root 2023-07-14T12:01:43.610 INFO:journalctl@ceph.mds.e.smithi093.stdout:Jul 14 12:01:43 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-e[153790]: ignoring --setgroup ceph since I am not root 2023-07-14T12:01:43.611 INFO:journalctl@ceph.mds.e.smithi093.stdout:Jul 14 12:01:43 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-e[153790]: starting mds.e at 2023-07-14T12:01:43.611 INFO:journalctl@ceph.mds.b.smithi093.stdout:Jul 14 12:01:43 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-b[154480]: ignoring --setuser ceph since I am not root 2023-07-14T12:01:43.612 INFO:journalctl@ceph.mds.b.smithi093.stdout:Jul 14 12:01:43 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-b[154480]: ignoring --setgroup ceph since I am not root 2023-07-14T12:01:43.612 INFO:journalctl@ceph.mds.b.smithi093.stdout:Jul 14 12:01:43 smithi093 ceph-mds[154504]: -- 172.21.15.93:0/1984200769 <== mon.2 v2:172.21.15.93:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55bdcf2b51e0 con 0x55bdcf28b000 2023-07-14T12:01:43.612 INFO:journalctl@ceph.mds.b.smithi093.stdout:Jul 14 12:01:43 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-b[154480]: starting mds.b at 2023-07-14T12:01:43.613 INFO:journalctl@ceph.mds.h.smithi093.stdout:Jul 14 12:01:43 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-h[154141]: ignoring --setuser ceph since I am not root 2023-07-14T12:01:43.613 INFO:journalctl@ceph.mds.h.smithi093.stdout:Jul 14 12:01:43 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-h[154141]: ignoring --setgroup ceph since I am not root 2023-07-14T12:01:43.613 INFO:journalctl@ceph.mds.h.smithi093.stdout:Jul 14 12:01:43 smithi093 ceph-mds[154166]: -- 172.21.15.93:0/1797921613 <== mon.2 v2:172.21.15.93:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561b981891e0 con 0x561b9815e800 2023-07-14T12:01:43.613 INFO:journalctl@ceph.mds.h.smithi093.stdout:Jul 14 12:01:43 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-h[154141]: starting mds.h at 2023-07-14T12:01:43.614 INFO:journalctl@ceph.mds.k.smithi093.stdout:Jul 14 12:01:43 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-k[153450]: ignoring --setuser ceph since I am not root 2023-07-14T12:01:43.614 INFO:journalctl@ceph.mds.k.smithi093.stdout:Jul 14 12:01:43 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-k[153450]: ignoring --setgroup ceph since I am not root 2023-07-14T12:01:43.614 INFO:journalctl@ceph.mds.k.smithi093.stdout:Jul 14 12:01:43 smithi093 ceph-mds[153474]: -- 172.21.15.93:0/1904581413 <== mon.2 v2:172.21.15.93:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555ac896b1e0 con 0x555ac8940c00 2023-07-14T12:01:43.615 INFO:journalctl@ceph.mds.k.smithi093.stdout:Jul 14 12:01:43 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-k[153450]: starting mds.k at 2023-07-14T12:01:43.634 INFO:journalctl@ceph.mds.i.smithi097.stdout:Jul 14 12:01:43 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-i[155680]: ignoring --setuser ceph since I am not root 2023-07-14T12:01:43.634 INFO:journalctl@ceph.mds.i.smithi097.stdout:Jul 14 12:01:43 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-i[155680]: ignoring --setgroup ceph since I am not root 2023-07-14T12:01:43.634 INFO:journalctl@ceph.mds.i.smithi097.stdout:Jul 14 12:01:43 smithi097 ceph-mds[155704]: -- 172.21.15.97:0/2458749860 <== mon.1 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c81fb751e0 con 0x55c81fb4ac00 2023-07-14T12:01:43.634 INFO:journalctl@ceph.mds.i.smithi097.stdout:Jul 14 12:01:43 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-i[155680]: starting mds.i at 2023-07-14T12:01:43.635 INFO:journalctl@ceph.mds.c.smithi097.stdout:Jul 14 12:01:43 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-c[155333]: ignoring --setuser ceph since I am not root 2023-07-14T12:01:43.635 INFO:journalctl@ceph.mds.c.smithi097.stdout:Jul 14 12:01:43 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-c[155333]: ignoring --setgroup ceph since I am not root 2023-07-14T12:01:43.636 INFO:journalctl@ceph.mds.c.smithi097.stdout:Jul 14 12:01:43 smithi097 ceph-mds[155357]: -- 172.21.15.97:0/216485840 <== mon.2 v2:172.21.15.93:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56436cc471e0 con 0x56436cc1c800 2023-07-14T12:01:43.636 INFO:journalctl@ceph.mds.c.smithi097.stdout:Jul 14 12:01:43 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-c[155333]: starting mds.c at 2023-07-14T12:01:43.637 INFO:journalctl@ceph.mds.f.smithi097.stdout:Jul 14 12:01:43 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-f[154995]: ignoring --setuser ceph since I am not root 2023-07-14T12:01:43.637 INFO:journalctl@ceph.mds.f.smithi097.stdout:Jul 14 12:01:43 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-f[154995]: ignoring --setgroup ceph since I am not root 2023-07-14T12:01:43.638 INFO:journalctl@ceph.mds.f.smithi097.stdout:Jul 14 12:01:43 smithi097 ceph-mds[155019]: -- 172.21.15.97:0/3184657981 <== mon.1 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560b783911e0 con 0x560b78367000 2023-07-14T12:01:43.638 INFO:journalctl@ceph.mds.f.smithi097.stdout:Jul 14 12:01:43 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-f[154995]: starting mds.f at 2023-07-14T12:01:43.638 INFO:journalctl@ceph.mds.l.smithi097.stdout:Jul 14 12:01:43 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-l[154654]: ignoring --setuser ceph since I am not root 2023-07-14T12:01:43.638 INFO:journalctl@ceph.mds.l.smithi097.stdout:Jul 14 12:01:43 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-l[154654]: ignoring --setgroup ceph since I am not root 2023-07-14T12:01:43.639 INFO:journalctl@ceph.mds.l.smithi097.stdout:Jul 14 12:01:43 smithi097 ceph-mds[154678]: -- 172.21.15.97:0/138442464 <== mon.1 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56082e7111e0 con 0x56082e6e6c00 2023-07-14T12:01:43.639 INFO:journalctl@ceph.mds.l.smithi097.stdout:Jul 14 12:01:43 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-l[154654]: starting mds.l at 2023-07-14T12:01:43.643 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:43 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mon-a[126315]: 2023-07-14T12:01:43.261+0000 7fd775af4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-14T12:01:43.644 INFO:journalctl@ceph.mds.g.smithi017.stdout:Jul 14 12:01:43 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-g[160796]: ignoring --setuser ceph since I am not root 2023-07-14T12:01:43.644 INFO:journalctl@ceph.mds.g.smithi017.stdout:Jul 14 12:01:43 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-g[160796]: ignoring --setgroup ceph since I am not root 2023-07-14T12:01:43.644 INFO:journalctl@ceph.mds.g.smithi017.stdout:Jul 14 12:01:43 smithi017 ceph-mds[160820]: -- 172.21.15.17:0/2940167243 <== mon.2 v2:172.21.15.93:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5619518ef1e0 con 0x5619518c4800 2023-07-14T12:01:43.644 INFO:journalctl@ceph.mds.g.smithi017.stdout:Jul 14 12:01:43 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-g[160796]: starting mds.g at 2023-07-14T12:01:43.645 INFO:journalctl@ceph.mds.a.smithi017.stdout:Jul 14 12:01:43 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-a[161183]: -6160> 2023-07-14T11:59:46.001+0000 7fcd1b40b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-14T12:01:43.645 INFO:journalctl@ceph.mds.a.smithi017.stdout:Jul 14 12:01:43 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-a[161183]: -3330> 2023-07-14T12:00:16.004+0000 7fcd1b40b700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-07-14T12:01:43.645 INFO:journalctl@ceph.mds.a.smithi017.stdout:Jul 14 12:01:43 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-a[161183]: ignoring --setuser ceph since I am not root 2023-07-14T12:01:43.645 INFO:journalctl@ceph.mds.a.smithi017.stdout:Jul 14 12:01:43 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-a[161183]: ignoring --setgroup ceph since I am not root 2023-07-14T12:01:43.646 INFO:journalctl@ceph.mds.a.smithi017.stdout:Jul 14 12:01:43 smithi017 ceph-mds[161207]: -- 172.21.15.17:0/179269284 <== mon.2 v2:172.21.15.93:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557b944d71e0 con 0x557b944acc00 2023-07-14T12:01:43.646 INFO:journalctl@ceph.mds.a.smithi017.stdout:Jul 14 12:01:43 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-a[161183]: starting mds.a at 2023-07-14T12:01:43.760 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:43.782+0000 7f2c4f3ff700 1 -- 172.21.15.17:0/1008224092 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2c48106a90 con 0x7f2c48100dd0 2023-07-14T12:01:44.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:43 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/3846934238' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-14T12:01:44.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:43 smithi093 ceph-mon[128922]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-07-14T12:01:44.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:43 smithi093 ceph-mon[128922]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-14T12:01:44.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:43 smithi093 ceph-mon[128922]: osdmap e93: 12 total, 12 up, 12 in 2023-07-14T12:01:44.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:43 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/3846934238' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-07-14T12:01:44.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:43 smithi093 ceph-mon[128922]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-07-14T12:01:44.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:43 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/1008224092' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-14T12:01:44.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:43 smithi093 ceph-mon[128922]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-14T12:01:44.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:43 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/3846934238' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-14T12:01:44.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:43 smithi097 ceph-mon[129313]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-07-14T12:01:44.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:43 smithi097 ceph-mon[129313]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-14T12:01:44.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:43 smithi097 ceph-mon[129313]: osdmap e93: 12 total, 12 up, 12 in 2023-07-14T12:01:44.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:43 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/3846934238' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-07-14T12:01:44.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:43 smithi097 ceph-mon[129313]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-07-14T12:01:44.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:43 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/1008224092' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-14T12:01:44.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:43 smithi097 ceph-mon[129313]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-14T12:01:44.143 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:43 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/3846934238' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-14T12:01:44.143 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:43 smithi017 ceph-mon[126339]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-07-14T12:01:44.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:43 smithi017 ceph-mon[126339]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-14T12:01:44.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:43 smithi017 ceph-mon[126339]: osdmap e93: 12 total, 12 up, 12 in 2023-07-14T12:01:44.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:43 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/3846934238' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-07-14T12:01:44.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:43 smithi017 ceph-mon[126339]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-07-14T12:01:44.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:43 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/1008224092' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-14T12:01:44.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:43 smithi017 ceph-mon[126339]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-14T12:01:44.248 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.269+0000 7f2c3e7fc700 1 -- 172.21.15.17:0/1008224092 <== mon.2 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2c38097310 con 0x7f2c48100dd0 2023-07-14T12:01:44.252 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.273+0000 7f2c4f3ff700 1 -- 172.21.15.17:0/1008224092 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f2c340a11d0 msgr2=0x7f2c340a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:44.252 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.273+0000 7f2c4f3ff700 1 --2- 172.21.15.17:0/1008224092 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f2c340a11d0 0x7f2c340a3680 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2c48100c30 tx=0x7f2c44009260 comp rx=0 tx=0).stop 2023-07-14T12:01:44.253 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.273+0000 7f2c4f3ff700 1 -- 172.21.15.17:0/1008224092 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c48100dd0 msgr2=0x7f2c48072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:44.253 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.273+0000 7f2c4f3ff700 1 --2- 172.21.15.17:0/1008224092 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c48100dd0 0x7f2c48072900 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2c3800c010 tx=0x7f2c38004730 comp rx=0 tx=0).stop 2023-07-14T12:01:44.253 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.274+0000 7f2c4f3ff700 1 -- 172.21.15.17:0/1008224092 shutdown_connections 2023-07-14T12:01:44.254 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.274+0000 7f2c4f3ff700 1 --2- 172.21.15.17:0/1008224092 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f2c340a11d0 0x7f2c340a3680 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:44.254 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.274+0000 7f2c4f3ff700 1 --2- 172.21.15.17:0/1008224092 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c48101760 0x7f2c48072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:44.254 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.274+0000 7f2c4f3ff700 1 --2- 172.21.15.17:0/1008224092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c480ffbe0 0x7f2c480742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:44.254 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.274+0000 7f2c4f3ff700 1 --2- 172.21.15.17:0/1008224092 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c48100dd0 0x7f2c48072900 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:44.255 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.274+0000 7f2c4f3ff700 1 -- 172.21.15.17:0/1008224092 >> 172.21.15.17:0/1008224092 conn(0x7f2c480fb200 msgr2=0x7f2c480fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:44.255 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.275+0000 7f2c4f3ff700 1 -- 172.21.15.17:0/1008224092 shutdown_connections 2023-07-14T12:01:44.255 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.275+0000 7f2c4f3ff700 1 -- 172.21.15.17:0/1008224092 wait complete. 2023-07-14T12:01:44.276 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-07-14T12:01:44.488 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.509+0000 7fd883bff700 1 -- 172.21.15.17:0/770845557 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd87c0ffbe0 msgr2=0x7fd87c0ffff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:44.488 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.509+0000 7fd883bff700 1 --2- 172.21.15.17:0/770845557 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd87c0ffbe0 0x7fd87c0ffff0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd878009990 tx=0x7fd878009ca0 comp rx=0 tx=0).stop 2023-07-14T12:01:44.489 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.509+0000 7fd883bff700 1 -- 172.21.15.17:0/770845557 shutdown_connections 2023-07-14T12:01:44.489 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.509+0000 7fd883bff700 1 --2- 172.21.15.17:0/770845557 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd87c101760 0x7fd87c108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:44.489 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.509+0000 7fd883bff700 1 --2- 172.21.15.17:0/770845557 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd87c0ffbe0 0x7fd87c0ffff0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:44.489 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.509+0000 7fd883bff700 1 --2- 172.21.15.17:0/770845557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd87c100dd0 0x7fd87c101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:44.490 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.509+0000 7fd883bff700 1 -- 172.21.15.17:0/770845557 >> 172.21.15.17:0/770845557 conn(0x7fd87c0fb200 msgr2=0x7fd87c0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:44.490 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.509+0000 7fd883bff700 1 -- 172.21.15.17:0/770845557 shutdown_connections 2023-07-14T12:01:44.490 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.509+0000 7fd883bff700 1 -- 172.21.15.17:0/770845557 wait complete. 2023-07-14T12:01:44.492 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.510+0000 7fd883bff700 1 Processor -- start 2023-07-14T12:01:44.492 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.510+0000 7fd883bff700 1 -- start start 2023-07-14T12:01:44.492 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.511+0000 7fd883bff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd87c0ffbe0 0x7fd87c197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:44.492 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.511+0000 7fd883bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd87c100dd0 0x7fd87c198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:44.493 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.511+0000 7fd883bff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd87c101760 0x7fd87c19f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:44.493 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.511+0000 7fd883bff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd87c19f760 con 0x7fd87c0ffbe0 2023-07-14T12:01:44.493 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.511+0000 7fd883bff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd87c19f8a0 con 0x7fd87c101760 2023-07-14T12:01:44.493 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.511+0000 7fd883bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd87c19fa30 con 0x7fd87c100dd0 2023-07-14T12:01:44.493 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.511+0000 7fd88199b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd87c0ffbe0 0x7fd87c197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:44.494 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.511+0000 7fd88199b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd87c0ffbe0 0x7fd87c197be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52288/0 (socket says 172.21.15.17:52288) 2023-07-14T12:01:44.494 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.511+0000 7fd88199b700 1 -- 172.21.15.17:0/170573951 learned_addr learned my addr 172.21.15.17:0/170573951 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:44.494 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.511+0000 7fd88219c700 1 --2- 172.21.15.17:0/170573951 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd87c101760 0x7fd87c19f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:44.494 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.511+0000 7fd88119a700 1 --2- 172.21.15.17:0/170573951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd87c100dd0 0x7fd87c198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:44.494 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.511+0000 7fd88199b700 1 -- 172.21.15.17:0/170573951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd87c100dd0 msgr2=0x7fd87c198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:44.495 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.511+0000 7fd88199b700 1 --2- 172.21.15.17:0/170573951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd87c100dd0 0x7fd87c198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:44.495 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.511+0000 7fd88199b700 1 -- 172.21.15.17:0/170573951 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd87c101760 msgr2=0x7fd87c19f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:44.495 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.511+0000 7fd88199b700 1 --2- 172.21.15.17:0/170573951 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd87c101760 0x7fd87c19f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:44.495 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.511+0000 7fd88199b700 1 -- 172.21.15.17:0/170573951 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8780095f0 con 0x7fd87c0ffbe0 2023-07-14T12:01:44.495 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.512+0000 7fd88199b700 1 --2- 172.21.15.17:0/170573951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd87c0ffbe0 0x7fd87c197be0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd878000c00 tx=0x7fd878005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:44.495 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.512+0000 7fd872ffd700 1 -- 172.21.15.17:0/170573951 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87801d070 con 0x7fd87c0ffbe0 2023-07-14T12:01:44.496 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.512+0000 7fd872ffd700 1 -- 172.21.15.17:0/170573951 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd878003f60 con 0x7fd87c0ffbe0 2023-07-14T12:01:44.496 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.512+0000 7fd883bff700 1 -- 172.21.15.17:0/170573951 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd87c19fcb0 con 0x7fd87c0ffbe0 2023-07-14T12:01:44.496 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.512+0000 7fd872ffd700 1 -- 172.21.15.17:0/170573951 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd878022b50 con 0x7fd87c0ffbe0 2023-07-14T12:01:44.496 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.512+0000 7fd883bff700 1 -- 172.21.15.17:0/170573951 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd87c1a00d0 con 0x7fd87c0ffbe0 2023-07-14T12:01:44.497 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.513+0000 7fd883bff700 1 -- 172.21.15.17:0/170573951 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd87c1922c0 con 0x7fd87c0ffbe0 2023-07-14T12:01:44.497 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.515+0000 7fd872ffd700 1 -- 172.21.15.17:0/170573951 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd8780040d0 con 0x7fd87c0ffbe0 2023-07-14T12:01:44.498 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.516+0000 7fd872ffd700 1 --2- 172.21.15.17:0/170573951 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fd8680a1150 0x7fd8680a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:44.498 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.516+0000 7fd872ffd700 1 -- 172.21.15.17:0/170573951 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9508+0+0 (secure 0 0 0) 0x7fd8780ce510 con 0x7fd87c0ffbe0 2023-07-14T12:01:44.498 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.517+0000 7fd88119a700 1 --2- 172.21.15.17:0/170573951 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fd8680a1150 0x7fd8680a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:44.498 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.517+0000 7fd88119a700 1 --2- 172.21.15.17:0/170573951 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fd8680a1150 0x7fd8680a3600 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd86c007be0 tx=0x7fd86c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:44.499 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.521+0000 7fd872ffd700 1 -- 172.21.15.17:0/170573951 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd878097790 con 0x7fd87c0ffbe0 2023-07-14T12:01:44.710 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:44.732+0000 7fd883bff700 1 -- 172.21.15.17:0/170573951 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd87c02c850 con 0x7fd87c0ffbe0 2023-07-14T12:01:45.110 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:44 smithi093 ceph-mon[128922]: pgmap v342: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2023-07-14T12:01:45.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:44 smithi093 ceph-mon[128922]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-07-14T12:01:45.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:44 smithi093 ceph-mon[128922]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-07-14T12:01:45.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:44 smithi093 ceph-mon[128922]: Cluster is now healthy 2023-07-14T12:01:45.111 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:44 smithi093 ceph-mon[128922]: mds.? [v2:172.21.15.93:6836/1466526707,v1:172.21.15.93:6837/1466526707] up:boot 2023-07-14T12:01:45.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:44 smithi093 ceph-mon[128922]: mds.? [v2:172.21.15.97:6833/506837274,v1:172.21.15.97:6835/506837274] up:boot 2023-07-14T12:01:45.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:44 smithi093 ceph-mon[128922]: mds.? [v2:172.21.15.93:6838/1726199931,v1:172.21.15.93:6839/1726199931] up:boot 2023-07-14T12:01:45.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:44 smithi093 ceph-mon[128922]: mds.? [v2:172.21.15.97:6836/703964876,v1:172.21.15.97:6837/703964876] up:boot 2023-07-14T12:01:45.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:44 smithi093 ceph-mon[128922]: mds.? [v2:172.21.15.97:6838/1751802037,v1:172.21.15.97:6839/1751802037] up:boot 2023-07-14T12:01:45.112 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:44 smithi093 ceph-mon[128922]: mds.? [v2:172.21.15.93:6832/870724901,v1:172.21.15.93:6835/870724901] up:boot 2023-07-14T12:01:45.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:44 smithi093 ceph-mon[128922]: mds.? [v2:172.21.15.93:6833/1704581257,v1:172.21.15.93:6834/1704581257] up:boot 2023-07-14T12:01:45.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:44 smithi093 ceph-mon[128922]: mds.? [v2:172.21.15.97:6832/1439645503,v1:172.21.15.97:6834/1439645503] up:boot 2023-07-14T12:01:45.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:44 smithi093 ceph-mon[128922]: mds.? [v2:172.21.15.17:6838/3002706839,v1:172.21.15.17:6839/3002706839] up:boot 2023-07-14T12:01:45.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:44 smithi093 ceph-mon[128922]: mds.? [v2:172.21.15.17:6840/3259919709,v1:172.21.15.17:6841/3259919709] up:boot 2023-07-14T12:01:45.113 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:44 smithi093 ceph-mon[128922]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-07-14T12:01:45.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:44 smithi093 ceph-mon[128922]: fsmap 12 up:standby 2023-07-14T12:01:45.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:44 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-14T12:01:45.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:44 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-14T12:01:45.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:44 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-14T12:01:45.114 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:44 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-14T12:01:45.115 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:44 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-14T12:01:45.115 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:44 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-14T12:01:45.115 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:44 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-14T12:01:45.115 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:44 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-14T12:01:45.115 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:44 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-14T12:01:45.116 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:44 smithi093 ceph-mon[128922]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-14T12:01:45.116 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:44 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/170573951' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T12:01:45.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:44 smithi097 ceph-mon[129313]: pgmap v342: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2023-07-14T12:01:45.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:44 smithi097 ceph-mon[129313]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-07-14T12:01:45.134 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:44 smithi097 ceph-mon[129313]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-07-14T12:01:45.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:44 smithi097 ceph-mon[129313]: Cluster is now healthy 2023-07-14T12:01:45.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:44 smithi097 ceph-mon[129313]: mds.? [v2:172.21.15.93:6836/1466526707,v1:172.21.15.93:6837/1466526707] up:boot 2023-07-14T12:01:45.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:44 smithi097 ceph-mon[129313]: mds.? [v2:172.21.15.97:6833/506837274,v1:172.21.15.97:6835/506837274] up:boot 2023-07-14T12:01:45.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:44 smithi097 ceph-mon[129313]: mds.? [v2:172.21.15.93:6838/1726199931,v1:172.21.15.93:6839/1726199931] up:boot 2023-07-14T12:01:45.135 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:44 smithi097 ceph-mon[129313]: mds.? [v2:172.21.15.97:6836/703964876,v1:172.21.15.97:6837/703964876] up:boot 2023-07-14T12:01:45.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:44 smithi097 ceph-mon[129313]: mds.? [v2:172.21.15.97:6838/1751802037,v1:172.21.15.97:6839/1751802037] up:boot 2023-07-14T12:01:45.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:44 smithi097 ceph-mon[129313]: mds.? [v2:172.21.15.93:6832/870724901,v1:172.21.15.93:6835/870724901] up:boot 2023-07-14T12:01:45.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:44 smithi097 ceph-mon[129313]: mds.? [v2:172.21.15.93:6833/1704581257,v1:172.21.15.93:6834/1704581257] up:boot 2023-07-14T12:01:45.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:44 smithi097 ceph-mon[129313]: mds.? [v2:172.21.15.97:6832/1439645503,v1:172.21.15.97:6834/1439645503] up:boot 2023-07-14T12:01:45.136 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:44 smithi097 ceph-mon[129313]: mds.? [v2:172.21.15.17:6838/3002706839,v1:172.21.15.17:6839/3002706839] up:boot 2023-07-14T12:01:45.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:44 smithi097 ceph-mon[129313]: mds.? [v2:172.21.15.17:6840/3259919709,v1:172.21.15.17:6841/3259919709] up:boot 2023-07-14T12:01:45.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:44 smithi097 ceph-mon[129313]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-07-14T12:01:45.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:44 smithi097 ceph-mon[129313]: fsmap 12 up:standby 2023-07-14T12:01:45.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:44 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-14T12:01:45.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:44 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-14T12:01:45.137 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:44 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-14T12:01:45.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:44 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-14T12:01:45.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:44 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-14T12:01:45.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:44 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-14T12:01:45.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:44 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-14T12:01:45.138 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:44 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-14T12:01:45.139 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:44 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-14T12:01:45.139 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:44 smithi097 ceph-mon[129313]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-14T12:01:45.139 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:44 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/170573951' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T12:01:45.143 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:44 smithi017 ceph-mon[126339]: pgmap v342: 129 pgs: 129 active+clean; 586 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2023-07-14T12:01:45.143 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:44 smithi017 ceph-mon[126339]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-07-14T12:01:45.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:44 smithi017 ceph-mon[126339]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-07-14T12:01:45.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:44 smithi017 ceph-mon[126339]: Cluster is now healthy 2023-07-14T12:01:45.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:44 smithi017 ceph-mon[126339]: mds.? [v2:172.21.15.93:6836/1466526707,v1:172.21.15.93:6837/1466526707] up:boot 2023-07-14T12:01:45.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:44 smithi017 ceph-mon[126339]: mds.? [v2:172.21.15.97:6833/506837274,v1:172.21.15.97:6835/506837274] up:boot 2023-07-14T12:01:45.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:44 smithi017 ceph-mon[126339]: mds.? [v2:172.21.15.93:6838/1726199931,v1:172.21.15.93:6839/1726199931] up:boot 2023-07-14T12:01:45.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:44 smithi017 ceph-mon[126339]: mds.? [v2:172.21.15.97:6836/703964876,v1:172.21.15.97:6837/703964876] up:boot 2023-07-14T12:01:45.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:44 smithi017 ceph-mon[126339]: mds.? [v2:172.21.15.97:6838/1751802037,v1:172.21.15.97:6839/1751802037] up:boot 2023-07-14T12:01:45.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:44 smithi017 ceph-mon[126339]: mds.? [v2:172.21.15.93:6832/870724901,v1:172.21.15.93:6835/870724901] up:boot 2023-07-14T12:01:45.145 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:44 smithi017 ceph-mon[126339]: mds.? [v2:172.21.15.93:6833/1704581257,v1:172.21.15.93:6834/1704581257] up:boot 2023-07-14T12:01:45.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:44 smithi017 ceph-mon[126339]: mds.? [v2:172.21.15.97:6832/1439645503,v1:172.21.15.97:6834/1439645503] up:boot 2023-07-14T12:01:45.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:44 smithi017 ceph-mon[126339]: mds.? [v2:172.21.15.17:6838/3002706839,v1:172.21.15.17:6839/3002706839] up:boot 2023-07-14T12:01:45.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:44 smithi017 ceph-mon[126339]: mds.? [v2:172.21.15.17:6840/3259919709,v1:172.21.15.17:6841/3259919709] up:boot 2023-07-14T12:01:45.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:44 smithi017 ceph-mon[126339]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-07-14T12:01:45.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:44 smithi017 ceph-mon[126339]: fsmap 12 up:standby 2023-07-14T12:01:45.146 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:44 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-14T12:01:45.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:44 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-14T12:01:45.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:44 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-14T12:01:45.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:44 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-14T12:01:45.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:44 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-14T12:01:45.147 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:44 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-14T12:01:45.148 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:44 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-14T12:01:45.148 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:44 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-14T12:01:45.148 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:44 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-14T12:01:45.148 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:44 smithi017 ceph-mon[126339]: from='mgr.14152 172.21.15.17:0/3166094080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-14T12:01:45.148 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:44 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/170573951' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T12:01:45.470 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.490+0000 7fd872ffd700 1 -- 172.21.15.17:0/170573951 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v94) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd87809adb0 con 0x7fd87c0ffbe0 2023-07-14T12:01:45.474 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.494+0000 7fd883bff700 1 -- 172.21.15.17:0/170573951 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fd8680a1150 msgr2=0x7fd8680a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:45.474 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.494+0000 7fd883bff700 1 --2- 172.21.15.17:0/170573951 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fd8680a1150 0x7fd8680a3600 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd86c007be0 tx=0x7fd86c0073d0 comp rx=0 tx=0).stop 2023-07-14T12:01:45.474 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.494+0000 7fd883bff700 1 -- 172.21.15.17:0/170573951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd87c0ffbe0 msgr2=0x7fd87c197be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:45.474 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.494+0000 7fd883bff700 1 --2- 172.21.15.17:0/170573951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd87c0ffbe0 0x7fd87c197be0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd878000c00 tx=0x7fd878005740 comp rx=0 tx=0).stop 2023-07-14T12:01:45.475 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.495+0000 7fd883bff700 1 -- 172.21.15.17:0/170573951 shutdown_connections 2023-07-14T12:01:45.475 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.495+0000 7fd883bff700 1 --2- 172.21.15.17:0/170573951 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fd8680a1150 0x7fd8680a3600 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:45.475 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.495+0000 7fd883bff700 1 --2- 172.21.15.17:0/170573951 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd87c101760 0x7fd87c19f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:45.475 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.495+0000 7fd883bff700 1 --2- 172.21.15.17:0/170573951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd87c0ffbe0 0x7fd87c197be0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:45.475 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.495+0000 7fd883bff700 1 --2- 172.21.15.17:0/170573951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd87c100dd0 0x7fd87c198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:45.476 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.495+0000 7fd883bff700 1 -- 172.21.15.17:0/170573951 >> 172.21.15.17:0/170573951 conn(0x7fd87c0fb200 msgr2=0x7fd87c1066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:45.476 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.495+0000 7fd883bff700 1 -- 172.21.15.17:0/170573951 shutdown_connections 2023-07-14T12:01:45.476 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.495+0000 7fd883bff700 1 -- 172.21.15.17:0/170573951 wait complete. 2023-07-14T12:01:45.476 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' removed 2023-07-14T12:01:45.496 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-07-14T12:01:45.715 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.736+0000 7fea3f9ff700 1 -- 172.21.15.17:0/848346520 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea38100dd0 msgr2=0x7fea38101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:45.715 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.736+0000 7fea3f9ff700 1 --2- 172.21.15.17:0/848346520 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea38100dd0 0x7fea38101220 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fea28009990 tx=0x7fea28009ca0 comp rx=0 tx=0).stop 2023-07-14T12:01:45.715 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.736+0000 7fea3f9ff700 1 -- 172.21.15.17:0/848346520 shutdown_connections 2023-07-14T12:01:45.716 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.736+0000 7fea3f9ff700 1 --2- 172.21.15.17:0/848346520 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea38101760 0x7fea38108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:45.716 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.736+0000 7fea3f9ff700 1 --2- 172.21.15.17:0/848346520 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea380ffbe0 0x7fea380ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:45.716 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.736+0000 7fea3f9ff700 1 --2- 172.21.15.17:0/848346520 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea38100dd0 0x7fea38101220 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:45.716 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.736+0000 7fea3f9ff700 1 -- 172.21.15.17:0/848346520 >> 172.21.15.17:0/848346520 conn(0x7fea380fb200 msgr2=0x7fea380fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:45.716 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.736+0000 7fea3f9ff700 1 -- 172.21.15.17:0/848346520 shutdown_connections 2023-07-14T12:01:45.717 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.736+0000 7fea3f9ff700 1 -- 172.21.15.17:0/848346520 wait complete. 2023-07-14T12:01:45.718 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.737+0000 7fea3f9ff700 1 Processor -- start 2023-07-14T12:01:45.719 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.737+0000 7fea3f9ff700 1 -- start start 2023-07-14T12:01:45.719 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.737+0000 7fea3f9ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea380ffbe0 0x7fea380742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:45.719 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.737+0000 7fea3f9ff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea38100dd0 0x7fea38072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:45.719 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.737+0000 7fea3f9ff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea38101760 0x7fea38072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:45.719 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.737+0000 7fea3f9ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea380735e0 con 0x7fea380ffbe0 2023-07-14T12:01:45.720 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.737+0000 7fea3f9ff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea38073750 con 0x7fea38101760 2023-07-14T12:01:45.720 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.737+0000 7fea3f9ff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3819fa10 con 0x7fea38100dd0 2023-07-14T12:01:45.720 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.738+0000 7fea3d79b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea380ffbe0 0x7fea380742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:45.720 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.738+0000 7fea3d79b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea380ffbe0 0x7fea380742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52310/0 (socket says 172.21.15.17:52310) 2023-07-14T12:01:45.720 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.738+0000 7fea3df9c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea38101760 0x7fea38072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:45.721 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.738+0000 7fea3d79b700 1 -- 172.21.15.17:0/4293164903 learned_addr learned my addr 172.21.15.17:0/4293164903 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:45.721 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.738+0000 7fea3cf9a700 1 --2- 172.21.15.17:0/4293164903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea38100dd0 0x7fea38072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:45.721 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.738+0000 7fea3d79b700 1 -- 172.21.15.17:0/4293164903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea38100dd0 msgr2=0x7fea38072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:45.721 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.738+0000 7fea3d79b700 1 --2- 172.21.15.17:0/4293164903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea38100dd0 0x7fea38072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:45.721 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.738+0000 7fea3d79b700 1 -- 172.21.15.17:0/4293164903 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea38101760 msgr2=0x7fea38072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:45.722 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.738+0000 7fea3d79b700 1 --2- 172.21.15.17:0/4293164903 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea38101760 0x7fea38072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:45.722 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.738+0000 7fea3d79b700 1 -- 172.21.15.17:0/4293164903 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea280095f0 con 0x7fea380ffbe0 2023-07-14T12:01:45.722 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.738+0000 7fea3d79b700 1 --2- 172.21.15.17:0/4293164903 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea380ffbe0 0x7fea380742c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fea3400e970 tx=0x7fea3400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:45.722 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.738+0000 7fea2e7fc700 1 -- 172.21.15.17:0/4293164903 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea34009790 con 0x7fea380ffbe0 2023-07-14T12:01:45.722 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.739+0000 7fea2e7fc700 1 -- 172.21.15.17:0/4293164903 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea3400c7e0 con 0x7fea380ffbe0 2023-07-14T12:01:45.723 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.739+0000 7fea2e7fc700 1 -- 172.21.15.17:0/4293164903 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea34003c70 con 0x7fea380ffbe0 2023-07-14T12:01:45.723 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.739+0000 7fea3f9ff700 1 -- 172.21.15.17:0/4293164903 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea3819fcf0 con 0x7fea380ffbe0 2023-07-14T12:01:45.723 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.739+0000 7fea3f9ff700 1 -- 172.21.15.17:0/4293164903 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea381a01b0 con 0x7fea380ffbe0 2023-07-14T12:01:45.723 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.740+0000 7fea3f9ff700 1 -- 172.21.15.17:0/4293164903 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea38107120 con 0x7fea380ffbe0 2023-07-14T12:01:45.724 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.742+0000 7fea2e7fc700 1 -- 172.21.15.17:0/4293164903 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fea3400cc50 con 0x7fea380ffbe0 2023-07-14T12:01:45.724 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.743+0000 7fea2e7fc700 1 --2- 172.21.15.17:0/4293164903 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fea240a1150 0x7fea240a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:45.724 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.743+0000 7fea2e7fc700 1 -- 172.21.15.17:0/4293164903 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9012+0+0 (secure 0 0 0) 0x7fea34014070 con 0x7fea380ffbe0 2023-07-14T12:01:45.725 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.743+0000 7fea3cf9a700 1 --2- 172.21.15.17:0/4293164903 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fea240a1150 0x7fea240a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:45.725 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.744+0000 7fea3cf9a700 1 --2- 172.21.15.17:0/4293164903 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fea240a1150 0x7fea240a3600 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fea2800b2e0 tx=0x7fea28005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:45.725 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.746+0000 7fea2e7fc700 1 -- 172.21.15.17:0/4293164903 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fea340961f0 con 0x7fea380ffbe0 2023-07-14T12:01:45.935 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:45.956+0000 7fea3f9ff700 1 -- 172.21.15.17:0/4293164903 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fea3802c850 con 0x7fea380ffbe0 2023-07-14T12:01:46.475 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.496+0000 7fea2e7fc700 1 -- 172.21.15.17:0/4293164903 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v95) v1 ==== 169+0+0 (secure 0 0 0) 0x7fea34003dd0 con 0x7fea380ffbe0 2023-07-14T12:01:46.478 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.500+0000 7fea3f9ff700 1 -- 172.21.15.17:0/4293164903 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fea240a1150 msgr2=0x7fea240a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:46.479 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.500+0000 7fea3f9ff700 1 --2- 172.21.15.17:0/4293164903 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fea240a1150 0x7fea240a3600 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fea2800b2e0 tx=0x7fea28005fb0 comp rx=0 tx=0).stop 2023-07-14T12:01:46.479 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.500+0000 7fea3f9ff700 1 -- 172.21.15.17:0/4293164903 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea380ffbe0 msgr2=0x7fea380742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:46.479 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.500+0000 7fea3f9ff700 1 --2- 172.21.15.17:0/4293164903 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea380ffbe0 0x7fea380742c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fea3400e970 tx=0x7fea3400ec80 comp rx=0 tx=0).stop 2023-07-14T12:01:46.480 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.500+0000 7fea3f9ff700 1 -- 172.21.15.17:0/4293164903 shutdown_connections 2023-07-14T12:01:46.480 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.500+0000 7fea3f9ff700 1 --2- 172.21.15.17:0/4293164903 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7fea240a1150 0x7fea240a3600 secure :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fea2800b2e0 tx=0x7fea28005fb0 comp rx=0 tx=0).stop 2023-07-14T12:01:46.480 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.500+0000 7fea3f9ff700 1 --2- 172.21.15.17:0/4293164903 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea38101760 0x7fea38072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:46.481 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.500+0000 7fea3f9ff700 1 --2- 172.21.15.17:0/4293164903 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea380ffbe0 0x7fea380742c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:46.481 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.500+0000 7fea3f9ff700 1 --2- 172.21.15.17:0/4293164903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea38100dd0 0x7fea38072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:46.481 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.500+0000 7fea3f9ff700 1 -- 172.21.15.17:0/4293164903 >> 172.21.15.17:0/4293164903 conn(0x7fea380fb200 msgr2=0x7fea38104260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:46.481 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.500+0000 7fea3f9ff700 1 -- 172.21.15.17:0/4293164903 shutdown_connections 2023-07-14T12:01:46.481 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.501+0000 7fea3f9ff700 1 -- 172.21.15.17:0/4293164903 wait complete. 2023-07-14T12:01:46.482 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' removed 2023-07-14T12:01:46.500 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-07-14T12:01:46.711 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.732+0000 7f94f5dff700 1 -- 172.21.15.17:0/3292348539 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94f0101800 msgr2=0x7f94f01080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:46.711 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.732+0000 7f94f5dff700 1 --2- 172.21.15.17:0/3292348539 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94f0101800 0x7f94f01080a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f94e8009f90 tx=0x7f94e800b330 comp rx=0 tx=0).stop 2023-07-14T12:01:46.711 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.732+0000 7f94f5dff700 1 -- 172.21.15.17:0/3292348539 shutdown_connections 2023-07-14T12:01:46.712 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.732+0000 7f94f5dff700 1 --2- 172.21.15.17:0/3292348539 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94f0101800 0x7f94f01080a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:46.712 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.732+0000 7f94f5dff700 1 --2- 172.21.15.17:0/3292348539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94f00ffbc0 0x7f94f00fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:46.712 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.732+0000 7f94f5dff700 1 --2- 172.21.15.17:0/3292348539 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f94f0100e50 0x7f94f01012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:46.712 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.732+0000 7f94f5dff700 1 -- 172.21.15.17:0/3292348539 >> 172.21.15.17:0/3292348539 conn(0x7f94f00fb200 msgr2=0x7f94f00fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:46.713 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.732+0000 7f94f5dff700 1 -- 172.21.15.17:0/3292348539 shutdown_connections 2023-07-14T12:01:46.713 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.732+0000 7f94f5dff700 1 -- 172.21.15.17:0/3292348539 wait complete. 2023-07-14T12:01:46.714 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.733+0000 7f94f5dff700 1 Processor -- start 2023-07-14T12:01:46.715 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.734+0000 7f94f5dff700 1 -- start start 2023-07-14T12:01:46.715 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.734+0000 7f94f5dff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f94f00ffbc0 0x7f94f00742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:46.715 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.734+0000 7f94f5dff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94f0100e50 0x7f94f0072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:46.715 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.734+0000 7f94f5dff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94f0101800 0x7f94f0072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:46.715 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.734+0000 7f94f5dff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f00735e0 con 0x7f94f0100e50 2023-07-14T12:01:46.716 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.734+0000 7f94f5dff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f0073750 con 0x7f94f00ffbc0 2023-07-14T12:01:46.716 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.734+0000 7f94f5dff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f019fa40 con 0x7f94f0101800 2023-07-14T12:01:46.716 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.734+0000 7f94eeffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94f0100e50 0x7f94f0072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:46.716 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.734+0000 7f94eeffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94f0100e50 0x7f94f0072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52330/0 (socket says 172.21.15.17:52330) 2023-07-14T12:01:46.716 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.734+0000 7f94eeffd700 1 -- 172.21.15.17:0/548385904 learned_addr learned my addr 172.21.15.17:0/548385904 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-07-14T12:01:46.717 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.735+0000 7f94ef7fe700 1 --2- 172.21.15.17:0/548385904 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f94f00ffbc0 0x7f94f00742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:46.717 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.735+0000 7f94effff700 1 --2- 172.21.15.17:0/548385904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94f0101800 0x7f94f0072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:46.717 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.735+0000 7f94eeffd700 1 -- 172.21.15.17:0/548385904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94f0101800 msgr2=0x7f94f0072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:46.717 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.735+0000 7f94eeffd700 1 --2- 172.21.15.17:0/548385904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94f0101800 0x7f94f0072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:46.717 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.735+0000 7f94eeffd700 1 -- 172.21.15.17:0/548385904 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f94f00ffbc0 msgr2=0x7f94f00742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:46.718 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.735+0000 7f94eeffd700 1 --2- 172.21.15.17:0/548385904 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f94f00ffbc0 0x7f94f00742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:46.718 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.735+0000 7f94eeffd700 1 -- 172.21.15.17:0/548385904 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e8009c40 con 0x7f94f0100e50 2023-07-14T12:01:46.718 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.735+0000 7f94eeffd700 1 --2- 172.21.15.17:0/548385904 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94f0100e50 0x7f94f0072900 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f94e400e9a0 tx=0x7f94e400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:46.718 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.735+0000 7f94ecff9700 1 -- 172.21.15.17:0/548385904 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e4009790 con 0x7f94f0100e50 2023-07-14T12:01:46.719 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.735+0000 7f94ecff9700 1 -- 172.21.15.17:0/548385904 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94e400ce80 con 0x7f94f0100e50 2023-07-14T12:01:46.719 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.735+0000 7f94f5dff700 1 -- 172.21.15.17:0/548385904 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94f019fd20 con 0x7f94f0100e50 2023-07-14T12:01:46.719 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.735+0000 7f94ecff9700 1 -- 172.21.15.17:0/548385904 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e4018680 con 0x7f94f0100e50 2023-07-14T12:01:46.719 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.735+0000 7f94f5dff700 1 -- 172.21.15.17:0/548385904 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94f01a0220 con 0x7f94f0100e50 2023-07-14T12:01:46.720 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.737+0000 7f94f5dff700 1 -- 172.21.15.17:0/548385904 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94f010b3e0 con 0x7f94f0100e50 2023-07-14T12:01:46.720 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.738+0000 7f94ecff9700 1 -- 172.21.15.17:0/548385904 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f94e4010430 con 0x7f94f0100e50 2023-07-14T12:01:46.721 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.740+0000 7f94ecff9700 1 --2- 172.21.15.17:0/548385904 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f94d80a11d0 0x7f94d80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T12:01:46.721 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.740+0000 7f94ecff9700 1 -- 172.21.15.17:0/548385904 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8424+0+0 (secure 0 0 0) 0x7f94e4014070 con 0x7f94f0100e50 2023-07-14T12:01:46.721 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.740+0000 7f94ef7fe700 1 --2- 172.21.15.17:0/548385904 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f94d80a11d0 0x7f94d80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T12:01:46.721 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.741+0000 7f94ef7fe700 1 --2- 172.21.15.17:0/548385904 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f94d80a11d0 0x7f94d80a3680 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f94e0005fd0 tx=0x7f94e0009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T12:01:46.722 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.743+0000 7f94ecff9700 1 -- 172.21.15.17:0/548385904 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f94e4051790 con 0x7f94f0100e50 2023-07-14T12:01:46.860 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:46 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/170573951' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-07-14T12:01:46.861 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:46 smithi093 ceph-mon[128922]: osdmap e94: 12 total, 12 up, 12 in 2023-07-14T12:01:46.861 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:46 smithi093 ceph-mon[128922]: pgmap v344: 97 pgs: 97 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T12:01:46.861 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:46 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/4293164903' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T12:01:46.884 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:46 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/170573951' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-07-14T12:01:46.884 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:46 smithi097 ceph-mon[129313]: osdmap e94: 12 total, 12 up, 12 in 2023-07-14T12:01:46.884 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:46 smithi097 ceph-mon[129313]: pgmap v344: 97 pgs: 97 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T12:01:46.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:46 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/4293164903' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T12:01:46.893 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:46 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/170573951' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-07-14T12:01:46.893 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:46 smithi017 ceph-mon[126339]: osdmap e94: 12 total, 12 up, 12 in 2023-07-14T12:01:46.893 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:46 smithi017 ceph-mon[126339]: pgmap v344: 97 pgs: 97 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T12:01:46.894 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:46 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/4293164903' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T12:01:46.932 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:46.953+0000 7f94f5dff700 1 -- 172.21.15.17:0/548385904 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f94f010b570 con 0x7f94f0100e50 2023-07-14T12:01:47.483 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:47.504+0000 7f94ecff9700 1 -- 172.21.15.17:0/548385904 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v96) v1 ==== 178+0+0 (secure 0 0 0) 0x7f94e4099c30 con 0x7f94f0100e50 2023-07-14T12:01:47.487 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:47.508+0000 7f94f5dff700 1 -- 172.21.15.17:0/548385904 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f94d80a11d0 msgr2=0x7f94d80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:47.488 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:47.508+0000 7f94f5dff700 1 --2- 172.21.15.17:0/548385904 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f94d80a11d0 0x7f94d80a3680 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f94e0005fd0 tx=0x7f94e0009310 comp rx=0 tx=0).stop 2023-07-14T12:01:47.488 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:47.508+0000 7f94f5dff700 1 -- 172.21.15.17:0/548385904 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94f0100e50 msgr2=0x7f94f0072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T12:01:47.488 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:47.508+0000 7f94f5dff700 1 --2- 172.21.15.17:0/548385904 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94f0100e50 0x7f94f0072900 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f94e400e9a0 tx=0x7f94e400ecb0 comp rx=0 tx=0).stop 2023-07-14T12:01:47.489 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:47.509+0000 7f94f5dff700 1 -- 172.21.15.17:0/548385904 shutdown_connections 2023-07-14T12:01:47.489 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:47.509+0000 7f94f5dff700 1 --2- 172.21.15.17:0/548385904 >> [v2:172.21.15.17:6800/4237449858,v1:172.21.15.17:6801/4237449858] conn(0x7f94d80a11d0 0x7f94d80a3680 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:47.489 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:47.509+0000 7f94f5dff700 1 --2- 172.21.15.17:0/548385904 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f94f00ffbc0 0x7f94f00742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:47.490 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:47.509+0000 7f94f5dff700 1 --2- 172.21.15.17:0/548385904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94f0101800 0x7f94f0072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:47.490 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:47.509+0000 7f94f5dff700 1 --2- 172.21.15.17:0/548385904 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94f0100e50 0x7f94f0072900 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T12:01:47.490 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:47.509+0000 7f94f5dff700 1 -- 172.21.15.17:0/548385904 >> 172.21.15.17:0/548385904 conn(0x7f94f00fb200 msgr2=0x7f94f00fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T12:01:47.490 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:47.509+0000 7f94f5dff700 1 -- 172.21.15.17:0/548385904 shutdown_connections 2023-07-14T12:01:47.490 INFO:teuthology.orchestra.run.smithi017.stderr:2023-07-14T12:01:47.509+0000 7f94f5dff700 1 -- 172.21.15.17:0/548385904 wait complete. 2023-07-14T12:01:47.491 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data_ec' removed 2023-07-14T12:01:47.513 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-14T12:01:47.585 DEBUG:teuthology.orchestra.run.smithi093:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-14T12:01:47.640 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-14T12:01:47.698 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-07-14T12:01:47.699 DEBUG:teuthology.orchestra.run.smithi017:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-07-14T12:01:47.743 DEBUG:teuthology.orchestra.run.smithi093:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-07-14T12:01:47.764 DEBUG:teuthology.orchestra.run.smithi097:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-07-14T12:01:47.782 INFO:tasks.cephadm:Stopping all daemons... 2023-07-14T12:01:47.782 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-07-14T12:01:47.783 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mon.a 2023-07-14T12:01:47.845 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:47 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/4293164903' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-07-14T12:01:47.845 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:47 smithi017 ceph-mon[126339]: osdmap e95: 12 total, 12 up, 12 in 2023-07-14T12:01:47.845 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:47 smithi017 ceph-mon[126339]: from='client.? 172.21.15.17:0/548385904' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T12:01:47.860 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:47 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/4293164903' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-07-14T12:01:47.860 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:47 smithi093 ceph-mon[128922]: osdmap e95: 12 total, 12 up, 12 in 2023-07-14T12:01:47.861 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:47 smithi093 ceph-mon[128922]: from='client.? 172.21.15.17:0/548385904' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T12:01:47.884 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:47 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/4293164903' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-07-14T12:01:47.884 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:47 smithi097 ceph-mon[129313]: osdmap e95: 12 total, 12 up, 12 in 2023-07-14T12:01:47.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:47 smithi097 ceph-mon[129313]: from='client.? 172.21.15.17:0/548385904' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T12:01:48.144 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:47 smithi017 systemd[1]: Stopping Ceph mon.a for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T12:01:48.643 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:48 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mon-a[126315]: 2023-07-14T12:01:48.309+0000 7fd77b2ff700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T12:01:48.644 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:48 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mon-a[126315]: 2023-07-14T12:01:48.309+0000 7fd77b2ff700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-07-14T12:01:49.143 INFO:journalctl@ceph.mon.a.smithi017.stdout:Jul 14 12:01:48 smithi017 bash[175300]: ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mon-a 2023-07-14T12:01:49.303 DEBUG:teuthology.orchestra.run.smithi017:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mon.a.service' 2023-07-14T12:01:49.414 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T12:01:49.414 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-07-14T12:01:49.415 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-07-14T12:01:49.415 DEBUG:teuthology.orchestra.run.smithi093:> sudo systemctl stop ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mon.b 2023-07-14T12:01:49.860 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:49 smithi093 systemd[1]: Stopping Ceph mon.b for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T12:01:50.137 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:49 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mon-b[128898]: 2023-07-14T12:01:49.882+0000 7f2e578d4700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T12:01:50.138 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:49 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mon-b[128898]: 2023-07-14T12:01:49.882+0000 7f2e578d4700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-07-14T12:01:50.532 INFO:journalctl@ceph.mon.b.smithi093.stdout:Jul 14 12:01:50 smithi093 bash[156282]: ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mon-b 2023-07-14T12:01:50.539 DEBUG:teuthology.orchestra.run.smithi093:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mon.b.service' 2023-07-14T12:01:50.586 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T12:01:50.587 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-07-14T12:01:50.587 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-07-14T12:01:50.588 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mon.c 2023-07-14T12:01:50.885 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:50 smithi097 systemd[1]: Stopping Ceph mon.c for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T12:01:51.364 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:50 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mon-c[129289]: 2023-07-14T12:01:50.992+0000 7fb528f10700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T12:01:51.364 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:50 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mon-c[129289]: 2023-07-14T12:01:50.992+0000 7fb528f10700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-07-14T12:01:51.589 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mon.c.service' 2023-07-14T12:01:51.614 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:51 smithi097 bash[157594]: ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mon-c 2023-07-14T12:01:51.614 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:51 smithi097 systemd[1]: ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mon.c.service: Succeeded. 2023-07-14T12:01:51.615 INFO:journalctl@ceph.mon.c.smithi097.stdout:Jul 14 12:01:51 smithi097 systemd[1]: Stopped Ceph mon.c for b3c386cc-223c-11ee-9b33-001a4aab830c. 2023-07-14T12:01:51.665 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T12:01:51.666 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-07-14T12:01:51.666 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-07-14T12:01:51.667 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mgr.x 2023-07-14T12:01:52.143 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 12:01:51 smithi017 systemd[1]: Stopping Ceph mgr.x for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T12:01:52.469 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 12:01:52 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T12:01:52.224+0000 7f3553862700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T12:01:52.470 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 12:01:52 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x[126595]: 2023-07-14T12:01:52.224+0000 7f3553862700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-07-14T12:01:53.045 INFO:journalctl@ceph.mgr.x.smithi017.stdout:Jul 14 12:01:52 smithi017 bash[175816]: ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-x 2023-07-14T12:01:53.070 DEBUG:teuthology.orchestra.run.smithi017:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mgr.x.service' 2023-07-14T12:01:53.162 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T12:01:53.162 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-07-14T12:01:53.163 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-07-14T12:01:53.163 DEBUG:teuthology.orchestra.run.smithi093:> sudo systemctl stop ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mgr.y 2023-07-14T12:01:53.578 INFO:journalctl@ceph.mgr.y.smithi093.stdout:Jul 14 12:01:53 smithi093 systemd[1]: Stopping Ceph mgr.y for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T12:01:53.860 INFO:journalctl@ceph.mgr.y.smithi093.stdout:Jul 14 12:01:53 smithi093 bash[156569]: ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-y 2023-07-14T12:01:53.860 INFO:journalctl@ceph.mgr.y.smithi093.stdout:Jul 14 12:01:53 smithi093 systemd[1]: ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-07-14T12:01:53.953 DEBUG:teuthology.orchestra.run.smithi093:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mgr.y.service' 2023-07-14T12:01:53.991 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T12:01:53.991 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-07-14T12:01:53.992 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-07-14T12:01:53.992 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mgr.z 2023-07-14T12:01:54.383 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Jul 14 12:01:54 smithi097 systemd[1]: Stopping Ceph mgr.z for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T12:01:54.701 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Jul 14 12:01:54 smithi097 bash[157747]: ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mgr-z 2023-07-14T12:01:54.881 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Jul 14 12:01:54 smithi097 systemd[1]: ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mgr.z.service: Main process exited, code=exited, status=143/n/a 2023-07-14T12:01:54.885 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mgr.z.service' 2023-07-14T12:01:54.922 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T12:01:54.923 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-07-14T12:01:54.923 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-07-14T12:01:54.924 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@osd.0 2023-07-14T12:01:55.393 INFO:journalctl@ceph.osd.0.smithi017.stdout:Jul 14 12:01:55 smithi017 systemd[1]: Stopping Ceph osd.0 for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T12:01:55.893 INFO:journalctl@ceph.osd.0.smithi017.stdout:Jul 14 12:01:55 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-0[139449]: 2023-07-14T12:01:55.456+0000 7fdce264a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T12:01:55.894 INFO:journalctl@ceph.osd.0.smithi017.stdout:Jul 14 12:01:55 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-0[139449]: 2023-07-14T12:01:55.456+0000 7fdce264a700 -1 osd.0 96 *** Got signal Terminated *** 2023-07-14T12:01:55.894 INFO:journalctl@ceph.osd.0.smithi017.stdout:Jul 14 12:01:55 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-0[139449]: 2023-07-14T12:01:55.456+0000 7fdce264a700 -1 osd.0 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T12:02:01.643 INFO:journalctl@ceph.osd.0.smithi017.stdout:Jul 14 12:02:01 smithi017 bash[175965]: ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-0 2023-07-14T12:02:03.447 DEBUG:teuthology.orchestra.run.smithi017:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@osd.0.service' 2023-07-14T12:02:03.540 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T12:02:03.542 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-07-14T12:02:03.543 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-07-14T12:02:03.543 DEBUG:teuthology.orchestra.run.smithi093:> sudo systemctl stop ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@osd.1 2023-07-14T12:02:03.860 INFO:journalctl@ceph.osd.1.smithi093.stdout:Jul 14 12:02:03 smithi093 systemd[1]: Stopping Ceph osd.1 for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T12:02:04.360 INFO:journalctl@ceph.osd.1.smithi093.stdout:Jul 14 12:02:03 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-1[134027]: 2023-07-14T12:02:03.907+0000 7f1176370700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T12:02:04.360 INFO:journalctl@ceph.osd.1.smithi093.stdout:Jul 14 12:02:03 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-1[134027]: 2023-07-14T12:02:03.907+0000 7f1176370700 -1 osd.1 96 *** Got signal Terminated *** 2023-07-14T12:02:04.361 INFO:journalctl@ceph.osd.1.smithi093.stdout:Jul 14 12:02:03 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-1[134027]: 2023-07-14T12:02:03.907+0000 7f1176370700 -1 osd.1 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T12:02:09.860 INFO:journalctl@ceph.osd.1.smithi093.stdout:Jul 14 12:02:09 smithi093 bash[156716]: ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-1 2023-07-14T12:02:10.880 DEBUG:teuthology.orchestra.run.smithi093:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@osd.1.service' 2023-07-14T12:02:11.088 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T12:02:11.089 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-07-14T12:02:11.089 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-07-14T12:02:11.090 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@osd.2 2023-07-14T12:02:11.383 INFO:journalctl@ceph.osd.2.smithi097.stdout:Jul 14 12:02:11 smithi097 systemd[1]: Stopping Ceph osd.2 for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T12:02:11.884 INFO:journalctl@ceph.osd.2.smithi097.stdout:Jul 14 12:02:11 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-2[135091]: 2023-07-14T12:02:11.474+0000 7f954b3f9700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T12:02:11.884 INFO:journalctl@ceph.osd.2.smithi097.stdout:Jul 14 12:02:11 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-2[135091]: 2023-07-14T12:02:11.474+0000 7f954b3f9700 -1 osd.2 96 *** Got signal Terminated *** 2023-07-14T12:02:11.884 INFO:journalctl@ceph.osd.2.smithi097.stdout:Jul 14 12:02:11 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-2[135091]: 2023-07-14T12:02:11.474+0000 7f954b3f9700 -1 osd.2 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T12:02:17.383 INFO:journalctl@ceph.osd.2.smithi097.stdout:Jul 14 12:02:17 smithi097 bash[157895]: ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-2 2023-07-14T12:02:18.774 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@osd.2.service' 2023-07-14T12:02:18.814 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T12:02:18.815 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-07-14T12:02:18.815 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-07-14T12:02:18.816 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@osd.3 2023-07-14T12:02:19.144 INFO:journalctl@ceph.osd.3.smithi017.stdout:Jul 14 12:02:18 smithi017 systemd[1]: Stopping Ceph osd.3 for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T12:02:19.643 INFO:journalctl@ceph.osd.3.smithi017.stdout:Jul 14 12:02:19 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-3[144885]: 2023-07-14T12:02:19.387+0000 7f626d6f3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T12:02:19.644 INFO:journalctl@ceph.osd.3.smithi017.stdout:Jul 14 12:02:19 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-3[144885]: 2023-07-14T12:02:19.387+0000 7f626d6f3700 -1 osd.3 96 *** Got signal Terminated *** 2023-07-14T12:02:19.644 INFO:journalctl@ceph.osd.3.smithi017.stdout:Jul 14 12:02:19 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-3[144885]: 2023-07-14T12:02:19.387+0000 7f626d6f3700 -1 osd.3 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T12:02:22.393 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:21 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:21.968+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:57.178998+0000 front 2023-07-14T12:01:57.179119+0000 (oldest deadline 2023-07-14T12:02:21.279334+0000) 2023-07-14T12:02:22.634 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:22 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:22.280+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:02:22.860 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:22 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:22.422+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:02:23.241 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:22 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:22.921+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:57.178998+0000 front 2023-07-14T12:01:57.179119+0000 (oldest deadline 2023-07-14T12:02:21.279334+0000) 2023-07-14T12:02:23.633 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:23 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:23.317+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:02:23.643 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:23 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:23.262+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:02:23.860 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:23 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:23.396+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:02:24.143 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:23 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:23.903+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:57.178998+0000 front 2023-07-14T12:01:57.179119+0000 (oldest deadline 2023-07-14T12:02:21.279334+0000) 2023-07-14T12:02:24.633 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:24 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:24.284+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:02:24.643 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:24 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:24.300+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:02:24.860 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:24 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:24.424+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:02:25.253 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:24 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:24.915+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:57.178998+0000 front 2023-07-14T12:01:57.179119+0000 (oldest deadline 2023-07-14T12:02:21.279334+0000) 2023-07-14T12:02:25.633 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:25 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:25.249+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:02:25.643 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:25 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:25.333+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:02:25.643 INFO:journalctl@ceph.osd.3.smithi017.stdout:Jul 14 12:02:25 smithi017 bash[176255]: ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-3 2023-07-14T12:02:25.860 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:25 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:25.464+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:02:26.209 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:25 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:25.950+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:57.178998+0000 front 2023-07-14T12:01:57.179119+0000 (oldest deadline 2023-07-14T12:02:21.279334+0000) 2023-07-14T12:02:26.633 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:26 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:26.250+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:02:26.643 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:26 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:26.355+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:02:26.860 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:26 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:26.453+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:02:27.186 DEBUG:teuthology.orchestra.run.smithi017:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@osd.3.service' 2023-07-14T12:02:27.250 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:26 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:26.960+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:57.178998+0000 front 2023-07-14T12:01:57.179119+0000 (oldest deadline 2023-07-14T12:02:21.279334+0000) 2023-07-14T12:02:27.251 INFO:journalctl@ceph.osd.3.smithi017.stdout:Jul 14 12:02:27 smithi017 systemd[1]: ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@osd.3.service: Succeeded. 2023-07-14T12:02:27.251 INFO:journalctl@ceph.osd.3.smithi017.stdout:Jul 14 12:02:27 smithi017 systemd[1]: Stopped Ceph osd.3 for b3c386cc-223c-11ee-9b33-001a4aab830c. 2023-07-14T12:02:27.277 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T12:02:27.278 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-07-14T12:02:27.279 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-07-14T12:02:27.279 DEBUG:teuthology.orchestra.run.smithi093:> sudo systemctl stop ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@osd.4 2023-07-14T12:02:27.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:27 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:27.466+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:02:27.611 INFO:journalctl@ceph.osd.4.smithi093.stdout:Jul 14 12:02:27 smithi093 systemd[1]: Stopping Ceph osd.4 for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T12:02:27.634 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:27 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:27.217+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:02:27.643 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:27 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:27.347+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:02:28.110 INFO:journalctl@ceph.osd.4.smithi093.stdout:Jul 14 12:02:27 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-4[139553]: 2023-07-14T12:02:27.663+0000 7efe5b545700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T12:02:28.110 INFO:journalctl@ceph.osd.4.smithi093.stdout:Jul 14 12:02:27 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-4[139553]: 2023-07-14T12:02:27.663+0000 7efe5b545700 -1 osd.4 96 *** Got signal Terminated *** 2023-07-14T12:02:28.110 INFO:journalctl@ceph.osd.4.smithi093.stdout:Jul 14 12:02:27 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-4[139553]: 2023-07-14T12:02:27.663+0000 7efe5b545700 -1 osd.4 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T12:02:28.306 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:27 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:27.928+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:57.178998+0000 front 2023-07-14T12:01:57.179119+0000 (oldest deadline 2023-07-14T12:02:21.279334+0000) 2023-07-14T12:02:28.633 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:28 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:28.206+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:02:28.643 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:28 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:28.327+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:02:28.860 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:28 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:28.426+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:02:29.323 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:28 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:28.967+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:57.178998+0000 front 2023-07-14T12:01:57.179119+0000 (oldest deadline 2023-07-14T12:02:21.279334+0000) 2023-07-14T12:02:29.633 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:29 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:29.240+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:02:29.643 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:29 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:29.343+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:02:29.860 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:29 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:29.437+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:02:30.339 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:29 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:29.956+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:57.178998+0000 front 2023-07-14T12:01:57.179119+0000 (oldest deadline 2023-07-14T12:02:21.279334+0000) 2023-07-14T12:02:30.633 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:30 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:30.288+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:02:30.634 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:30 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:30.476+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:06.686677+0000 front 2023-07-14T12:02:06.686702+0000 (oldest deadline 2023-07-14T12:02:29.586617+0000) 2023-07-14T12:02:30.643 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:30 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:30.360+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:02:30.860 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:30 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:30.461+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:02:31.143 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:30 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:30.913+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:57.178998+0000 front 2023-07-14T12:01:57.179119+0000 (oldest deadline 2023-07-14T12:02:21.279334+0000) 2023-07-14T12:02:31.633 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:31 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:31.526+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:06.686677+0000 front 2023-07-14T12:02:06.686702+0000 (oldest deadline 2023-07-14T12:02:29.586617+0000) 2023-07-14T12:02:31.634 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:31 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:31.331+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:02:31.643 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:31 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:31.377+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:02:31.643 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:31 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:31.377+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:05.537048+0000 front 2023-07-14T12:02:05.536734+0000 (oldest deadline 2023-07-14T12:02:30.836845+0000) 2023-07-14T12:02:31.860 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:31 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:31.428+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:02:31.860 INFO:journalctl@ceph.osd.4.smithi093.stdout:Jul 14 12:02:31 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-4[139553]: 2023-07-14T12:02:31.422+0000 7efe55b37700 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:05.235523+0000 front 2023-07-14T12:02:05.235500+0000 (oldest deadline 2023-07-14T12:02:31.135655+0000) 2023-07-14T12:02:32.143 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:31 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:31.898+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:57.178998+0000 front 2023-07-14T12:01:57.179119+0000 (oldest deadline 2023-07-14T12:02:21.279334+0000) 2023-07-14T12:02:32.360 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:31 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:31.899+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.438626+0000 front 2023-07-14T12:02:07.438648+0000 (oldest deadline 2023-07-14T12:02:31.538779+0000) 2023-07-14T12:02:32.633 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:32 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:32.288+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:02:32.634 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:32 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:32.517+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:06.686677+0000 front 2023-07-14T12:02:06.686702+0000 (oldest deadline 2023-07-14T12:02:29.586617+0000) 2023-07-14T12:02:32.643 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:32 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:32.328+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:02:32.643 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:32 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:32.328+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:05.537048+0000 front 2023-07-14T12:02:05.536734+0000 (oldest deadline 2023-07-14T12:02:30.836845+0000) 2023-07-14T12:02:32.860 INFO:journalctl@ceph.osd.4.smithi093.stdout:Jul 14 12:02:32 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-4[139553]: 2023-07-14T12:02:32.434+0000 7efe55b37700 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:05.235523+0000 front 2023-07-14T12:02:05.235500+0000 (oldest deadline 2023-07-14T12:02:31.135655+0000) 2023-07-14T12:02:32.861 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:32 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:32.397+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:02:33.317 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:32 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:32.917+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:57.178998+0000 front 2023-07-14T12:01:57.179119+0000 (oldest deadline 2023-07-14T12:02:21.279334+0000) 2023-07-14T12:02:33.317 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:32 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:32.917+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:08.182103+0000 front 2023-07-14T12:02:08.182077+0000 (oldest deadline 2023-07-14T12:02:32.281763+0000) 2023-07-14T12:02:33.325 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:32 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:32.907+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.438626+0000 front 2023-07-14T12:02:07.438648+0000 (oldest deadline 2023-07-14T12:02:31.538779+0000) 2023-07-14T12:02:33.610 INFO:journalctl@ceph.osd.4.smithi093.stdout:Jul 14 12:02:33 smithi093 bash[156993]: ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-4 2023-07-14T12:02:33.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:33 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:33.371+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:02:33.633 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:33 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:33.508+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:06.686677+0000 front 2023-07-14T12:02:06.686702+0000 (oldest deadline 2023-07-14T12:02:29.586617+0000) 2023-07-14T12:02:33.634 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:33 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:33.288+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:02:33.643 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:33 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:33.337+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:02:33.643 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:33 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:33.337+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:05.537048+0000 front 2023-07-14T12:02:05.536734+0000 (oldest deadline 2023-07-14T12:02:30.836845+0000) 2023-07-14T12:02:34.104 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:33 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:33.871+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.438626+0000 front 2023-07-14T12:02:07.438648+0000 (oldest deadline 2023-07-14T12:02:31.538779+0000) 2023-07-14T12:02:34.143 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:33 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:33.901+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:57.178998+0000 front 2023-07-14T12:01:57.179119+0000 (oldest deadline 2023-07-14T12:02:21.279334+0000) 2023-07-14T12:02:34.144 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:33 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:33.901+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:08.182103+0000 front 2023-07-14T12:02:08.182077+0000 (oldest deadline 2023-07-14T12:02:32.281763+0000) 2023-07-14T12:02:34.360 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:34 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:34.325+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:02:34.530 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:34 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:34.256+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:02:34.643 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:34 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:34.313+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:02:34.643 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:34 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:34.313+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:05.537048+0000 front 2023-07-14T12:02:05.536734+0000 (oldest deadline 2023-07-14T12:02:30.836845+0000) 2023-07-14T12:02:34.883 DEBUG:teuthology.orchestra.run.smithi093:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@osd.4.service' 2023-07-14T12:02:34.884 INFO:journalctl@ceph.osd.5.smithi097.stdout:Jul 14 12:02:34 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-5[140631]: 2023-07-14T12:02:34.750+0000 7f924e560700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:08.395112+0000 front 2023-07-14T12:02:08.395126+0000 (oldest deadline 2023-07-14T12:02:34.295178+0000) 2023-07-14T12:02:34.885 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:34 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:34.550+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:06.686677+0000 front 2023-07-14T12:02:06.686702+0000 (oldest deadline 2023-07-14T12:02:29.586617+0000) 2023-07-14T12:02:34.924 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T12:02:34.925 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-07-14T12:02:34.926 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-07-14T12:02:34.926 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@osd.5 2023-07-14T12:02:35.143 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:34 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:34.889+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:57.178998+0000 front 2023-07-14T12:01:57.179119+0000 (oldest deadline 2023-07-14T12:02:21.279334+0000) 2023-07-14T12:02:35.143 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:34 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:34.889+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:08.182103+0000 front 2023-07-14T12:02:08.182077+0000 (oldest deadline 2023-07-14T12:02:32.281763+0000) 2023-07-14T12:02:35.278 INFO:journalctl@ceph.osd.5.smithi097.stdout:Jul 14 12:02:34 smithi097 systemd[1]: Stopping Ceph osd.5 for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T12:02:35.285 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:34 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:34.896+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.438626+0000 front 2023-07-14T12:02:07.438648+0000 (oldest deadline 2023-07-14T12:02:31.538779+0000) 2023-07-14T12:02:35.570 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:35 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:35.298+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:02:35.570 INFO:journalctl@ceph.osd.5.smithi097.stdout:Jul 14 12:02:35 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-5[140631]: 2023-07-14T12:02:35.344+0000 7f9251f6a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T12:02:35.571 INFO:journalctl@ceph.osd.5.smithi097.stdout:Jul 14 12:02:35 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-5[140631]: 2023-07-14T12:02:35.344+0000 7f9251f6a700 -1 osd.5 96 *** Got signal Terminated *** 2023-07-14T12:02:35.571 INFO:journalctl@ceph.osd.5.smithi097.stdout:Jul 14 12:02:35 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-5[140631]: 2023-07-14T12:02:35.344+0000 7f9251f6a700 -1 osd.5 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T12:02:35.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:35 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:35.306+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:02:35.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:35 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:35.306+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.378517+0000 front 2023-07-14T12:02:07.378538+0000 (oldest deadline 2023-07-14T12:02:34.979250+0000) 2023-07-14T12:02:35.643 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:35 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:35.349+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:02:35.644 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:35 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:35.349+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:05.537048+0000 front 2023-07-14T12:02:05.536734+0000 (oldest deadline 2023-07-14T12:02:30.836845+0000) 2023-07-14T12:02:35.883 INFO:journalctl@ceph.osd.5.smithi097.stdout:Jul 14 12:02:35 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-5[140631]: 2023-07-14T12:02:35.748+0000 7f924e560700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:08.395112+0000 front 2023-07-14T12:02:08.395126+0000 (oldest deadline 2023-07-14T12:02:34.295178+0000) 2023-07-14T12:02:35.884 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:35 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:35.591+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:06.686677+0000 front 2023-07-14T12:02:06.686702+0000 (oldest deadline 2023-07-14T12:02:29.586617+0000) 2023-07-14T12:02:36.257 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:35 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:35.936+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.438626+0000 front 2023-07-14T12:02:07.438648+0000 (oldest deadline 2023-07-14T12:02:31.538779+0000) 2023-07-14T12:02:36.312 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:35 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:35.918+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:57.178998+0000 front 2023-07-14T12:01:57.179119+0000 (oldest deadline 2023-07-14T12:02:21.279334+0000) 2023-07-14T12:02:36.312 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:35 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:35.918+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:08.182103+0000 front 2023-07-14T12:02:08.182077+0000 (oldest deadline 2023-07-14T12:02:32.281763+0000) 2023-07-14T12:02:36.555 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:36 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:36.314+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:02:36.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:36 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:36.278+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:02:36.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:36 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:36.278+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.378517+0000 front 2023-07-14T12:02:07.378538+0000 (oldest deadline 2023-07-14T12:02:34.979250+0000) 2023-07-14T12:02:36.646 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:36 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:36.332+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:02:37.061 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:36 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:36.332+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:05.537048+0000 front 2023-07-14T12:02:05.536734+0000 (oldest deadline 2023-07-14T12:02:30.836845+0000) 2023-07-14T12:02:37.062 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:36 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:36.576+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:06.686677+0000 front 2023-07-14T12:02:06.686702+0000 (oldest deadline 2023-07-14T12:02:29.586617+0000) 2023-07-14T12:02:37.063 INFO:journalctl@ceph.osd.5.smithi097.stdout:Jul 14 12:02:36 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-5[140631]: 2023-07-14T12:02:36.755+0000 7f924e560700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:08.395112+0000 front 2023-07-14T12:02:08.395126+0000 (oldest deadline 2023-07-14T12:02:34.295178+0000) 2023-07-14T12:02:37.230 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:36 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:36.932+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.438626+0000 front 2023-07-14T12:02:07.438648+0000 (oldest deadline 2023-07-14T12:02:31.538779+0000) 2023-07-14T12:02:37.277 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:36 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:36.938+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:57.178998+0000 front 2023-07-14T12:01:57.179119+0000 (oldest deadline 2023-07-14T12:02:21.279334+0000) 2023-07-14T12:02:37.277 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:36 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:36.938+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:08.182103+0000 front 2023-07-14T12:02:08.182077+0000 (oldest deadline 2023-07-14T12:02:32.281763+0000) 2023-07-14T12:02:37.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:37 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:37.250+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:02:37.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:37 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:37.250+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.378517+0000 front 2023-07-14T12:02:07.378538+0000 (oldest deadline 2023-07-14T12:02:34.979250+0000) 2023-07-14T12:02:37.633 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:37 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:37.350+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:02:37.634 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:37 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:37.560+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:06.686677+0000 front 2023-07-14T12:02:06.686702+0000 (oldest deadline 2023-07-14T12:02:29.586617+0000) 2023-07-14T12:02:37.635 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:37 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:37.560+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:11.887528+0000 front 2023-07-14T12:02:11.887582+0000 (oldest deadline 2023-07-14T12:02:37.187740+0000) 2023-07-14T12:02:37.643 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:37 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:37.297+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:02:37.644 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:37 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:37.297+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:05.537048+0000 front 2023-07-14T12:02:05.536734+0000 (oldest deadline 2023-07-14T12:02:30.836845+0000) 2023-07-14T12:02:38.133 INFO:journalctl@ceph.osd.5.smithi097.stdout:Jul 14 12:02:37 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-5[140631]: 2023-07-14T12:02:37.722+0000 7f924e560700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:08.395112+0000 front 2023-07-14T12:02:08.395126+0000 (oldest deadline 2023-07-14T12:02:34.295178+0000) 2023-07-14T12:02:38.238 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:37 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:37.942+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:57.178998+0000 front 2023-07-14T12:01:57.179119+0000 (oldest deadline 2023-07-14T12:02:21.279334+0000) 2023-07-14T12:02:38.238 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:37 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:37.942+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:08.182103+0000 front 2023-07-14T12:02:08.182077+0000 (oldest deadline 2023-07-14T12:02:32.281763+0000) 2023-07-14T12:02:38.252 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:37 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:37.945+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.438626+0000 front 2023-07-14T12:02:07.438648+0000 (oldest deadline 2023-07-14T12:02:31.538779+0000) 2023-07-14T12:02:38.253 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:37 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:37.946+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:11.539352+0000 front 2023-07-14T12:02:11.539314+0000 (oldest deadline 2023-07-14T12:02:37.439216+0000) 2023-07-14T12:02:38.584 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:38 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:38.340+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:02:38.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:38 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:38.272+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:02:38.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:38 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:38.272+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.378517+0000 front 2023-07-14T12:02:07.378538+0000 (oldest deadline 2023-07-14T12:02:34.979250+0000) 2023-07-14T12:02:38.643 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:38 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:38.258+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:02:38.643 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:38 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:38.258+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:05.537048+0000 front 2023-07-14T12:02:05.536734+0000 (oldest deadline 2023-07-14T12:02:30.836845+0000) 2023-07-14T12:02:38.883 INFO:journalctl@ceph.osd.5.smithi097.stdout:Jul 14 12:02:38 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-5[140631]: 2023-07-14T12:02:38.739+0000 7f924e560700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:08.395112+0000 front 2023-07-14T12:02:08.395126+0000 (oldest deadline 2023-07-14T12:02:34.295178+0000) 2023-07-14T12:02:38.884 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:38 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:38.605+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:06.686677+0000 front 2023-07-14T12:02:06.686702+0000 (oldest deadline 2023-07-14T12:02:29.586617+0000) 2023-07-14T12:02:38.884 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:38 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:38.605+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:11.887528+0000 front 2023-07-14T12:02:11.887582+0000 (oldest deadline 2023-07-14T12:02:37.187740+0000) 2023-07-14T12:02:39.211 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:38 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:38.928+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.438626+0000 front 2023-07-14T12:02:07.438648+0000 (oldest deadline 2023-07-14T12:02:31.538779+0000) 2023-07-14T12:02:39.212 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:38 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:38.928+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:11.539352+0000 front 2023-07-14T12:02:11.539314+0000 (oldest deadline 2023-07-14T12:02:37.439216+0000) 2023-07-14T12:02:39.393 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:39 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:39.215+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:02:39.394 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:39 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:39.215+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:05.537048+0000 front 2023-07-14T12:02:05.536734+0000 (oldest deadline 2023-07-14T12:02:30.836845+0000) 2023-07-14T12:02:39.394 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:39 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:39.215+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:13.738547+0000 front 2023-07-14T12:02:13.738532+0000 (oldest deadline 2023-07-14T12:02:39.038483+0000) 2023-07-14T12:02:39.394 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:38 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:38.977+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:57.178998+0000 front 2023-07-14T12:01:57.179119+0000 (oldest deadline 2023-07-14T12:02:21.279334+0000) 2023-07-14T12:02:39.395 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:38 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:38.977+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:08.182103+0000 front 2023-07-14T12:02:08.182077+0000 (oldest deadline 2023-07-14T12:02:32.281763+0000) 2023-07-14T12:02:39.395 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:38 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:38.977+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:12.282755+0000 front 2023-07-14T12:02:12.282706+0000 (oldest deadline 2023-07-14T12:02:38.182367+0000) 2023-07-14T12:02:39.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:39 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:39.232+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:02:39.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:39 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:39.232+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.378517+0000 front 2023-07-14T12:02:07.378538+0000 (oldest deadline 2023-07-14T12:02:34.979250+0000) 2023-07-14T12:02:39.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:39 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:39.232+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:14.979892+0000 front 2023-07-14T12:02:14.979966+0000 (oldest deadline 2023-07-14T12:02:39.079887+0000) 2023-07-14T12:02:39.633 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:39 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:39.620+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:06.686677+0000 front 2023-07-14T12:02:06.686702+0000 (oldest deadline 2023-07-14T12:02:29.586617+0000) 2023-07-14T12:02:39.634 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:39 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:39.620+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:11.887528+0000 front 2023-07-14T12:02:11.887582+0000 (oldest deadline 2023-07-14T12:02:37.187740+0000) 2023-07-14T12:02:39.634 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:39 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:39.381+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:02:40.133 INFO:journalctl@ceph.osd.5.smithi097.stdout:Jul 14 12:02:39 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-5[140631]: 2023-07-14T12:02:39.780+0000 7f924e560700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:08.395112+0000 front 2023-07-14T12:02:08.395126+0000 (oldest deadline 2023-07-14T12:02:34.295178+0000) 2023-07-14T12:02:40.134 INFO:journalctl@ceph.osd.5.smithi097.stdout:Jul 14 12:02:39 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-5[140631]: 2023-07-14T12:02:39.780+0000 7f924e560700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:14.295597+0000 front 2023-07-14T12:02:14.295624+0000 (oldest deadline 2023-07-14T12:02:38.995790+0000) 2023-07-14T12:02:40.249 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:39 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:39.971+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.438626+0000 front 2023-07-14T12:02:07.438648+0000 (oldest deadline 2023-07-14T12:02:31.538779+0000) 2023-07-14T12:02:40.249 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:39 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:39.971+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:11.539352+0000 front 2023-07-14T12:02:11.539314+0000 (oldest deadline 2023-07-14T12:02:37.439216+0000) 2023-07-14T12:02:40.393 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:40 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:40.025+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:57.178998+0000 front 2023-07-14T12:01:57.179119+0000 (oldest deadline 2023-07-14T12:02:21.279334+0000) 2023-07-14T12:02:40.393 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:40 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:40.025+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:08.182103+0000 front 2023-07-14T12:02:08.182077+0000 (oldest deadline 2023-07-14T12:02:32.281763+0000) 2023-07-14T12:02:40.394 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:40 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:40.025+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:12.282755+0000 front 2023-07-14T12:02:12.282706+0000 (oldest deadline 2023-07-14T12:02:38.182367+0000) 2023-07-14T12:02:40.394 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:40 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:40.174+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:02:40.394 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:40 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:40.174+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:05.537048+0000 front 2023-07-14T12:02:05.536734+0000 (oldest deadline 2023-07-14T12:02:30.836845+0000) 2023-07-14T12:02:40.394 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:40 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:40.174+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:13.738547+0000 front 2023-07-14T12:02:13.738532+0000 (oldest deadline 2023-07-14T12:02:39.038483+0000) 2023-07-14T12:02:40.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:40 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:40.269+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:02:40.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:40 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:40.269+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.378517+0000 front 2023-07-14T12:02:07.378538+0000 (oldest deadline 2023-07-14T12:02:34.979250+0000) 2023-07-14T12:02:40.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:40 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:40.269+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:14.979892+0000 front 2023-07-14T12:02:14.979966+0000 (oldest deadline 2023-07-14T12:02:39.079887+0000) 2023-07-14T12:02:40.884 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:40 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:40.609+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:06.686677+0000 front 2023-07-14T12:02:06.686702+0000 (oldest deadline 2023-07-14T12:02:29.586617+0000) 2023-07-14T12:02:40.884 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:40 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:40.609+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:11.887528+0000 front 2023-07-14T12:02:11.887582+0000 (oldest deadline 2023-07-14T12:02:37.187740+0000) 2023-07-14T12:02:40.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:40 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:40.423+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:02:41.282 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:40 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:40.977+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.438626+0000 front 2023-07-14T12:02:07.438648+0000 (oldest deadline 2023-07-14T12:02:31.538779+0000) 2023-07-14T12:02:41.283 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:40 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:40.977+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:11.539352+0000 front 2023-07-14T12:02:11.539314+0000 (oldest deadline 2023-07-14T12:02:37.439216+0000) 2023-07-14T12:02:41.383 INFO:journalctl@ceph.osd.5.smithi097.stdout:Jul 14 12:02:41 smithi097 bash[158169]: ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-5 2023-07-14T12:02:41.393 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:41 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:41.139+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:02:41.394 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:41 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:41.139+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:05.537048+0000 front 2023-07-14T12:02:05.536734+0000 (oldest deadline 2023-07-14T12:02:30.836845+0000) 2023-07-14T12:02:41.394 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:41 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:41.139+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:13.738547+0000 front 2023-07-14T12:02:13.738532+0000 (oldest deadline 2023-07-14T12:02:39.038483+0000) 2023-07-14T12:02:41.394 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:41 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:41.041+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:57.178998+0000 front 2023-07-14T12:01:57.179119+0000 (oldest deadline 2023-07-14T12:02:21.279334+0000) 2023-07-14T12:02:41.395 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:41 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:41.041+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:08.182103+0000 front 2023-07-14T12:02:08.182077+0000 (oldest deadline 2023-07-14T12:02:32.281763+0000) 2023-07-14T12:02:41.395 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:41 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:41.041+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:12.282755+0000 front 2023-07-14T12:02:12.282706+0000 (oldest deadline 2023-07-14T12:02:38.182367+0000) 2023-07-14T12:02:41.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:41 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:41.303+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:02:41.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:41 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:41.303+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.378517+0000 front 2023-07-14T12:02:07.378538+0000 (oldest deadline 2023-07-14T12:02:34.979250+0000) 2023-07-14T12:02:41.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:41 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:41.303+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:14.979892+0000 front 2023-07-14T12:02:14.979966+0000 (oldest deadline 2023-07-14T12:02:39.079887+0000) 2023-07-14T12:02:41.763 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:41 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:41.472+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:02:41.763 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:41 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:41.472+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:15.551211+0000 front 2023-07-14T12:02:15.551186+0000 (oldest deadline 2023-07-14T12:02:40.851263+0000) 2023-07-14T12:02:41.764 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:41 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:41.628+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:06.686677+0000 front 2023-07-14T12:02:06.686702+0000 (oldest deadline 2023-07-14T12:02:29.586617+0000) 2023-07-14T12:02:41.764 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:41 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:41.628+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:11.887528+0000 front 2023-07-14T12:02:11.887582+0000 (oldest deadline 2023-07-14T12:02:37.187740+0000) 2023-07-14T12:02:42.272 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:41 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:41.993+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.438626+0000 front 2023-07-14T12:02:07.438648+0000 (oldest deadline 2023-07-14T12:02:31.538779+0000) 2023-07-14T12:02:42.273 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:41 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:41.993+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:11.539352+0000 front 2023-07-14T12:02:11.539314+0000 (oldest deadline 2023-07-14T12:02:37.439216+0000) 2023-07-14T12:02:42.393 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:42 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:42.138+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:02:42.394 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:42 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:42.138+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:05.537048+0000 front 2023-07-14T12:02:05.536734+0000 (oldest deadline 2023-07-14T12:02:30.836845+0000) 2023-07-14T12:02:42.394 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:42 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:42.138+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:13.738547+0000 front 2023-07-14T12:02:13.738532+0000 (oldest deadline 2023-07-14T12:02:39.038483+0000) 2023-07-14T12:02:42.395 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:42 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:42.071+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:57.178998+0000 front 2023-07-14T12:01:57.179119+0000 (oldest deadline 2023-07-14T12:02:21.279334+0000) 2023-07-14T12:02:42.395 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:42 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:42.071+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:08.182103+0000 front 2023-07-14T12:02:08.182077+0000 (oldest deadline 2023-07-14T12:02:32.281763+0000) 2023-07-14T12:02:42.395 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:42 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:42.071+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:12.282755+0000 front 2023-07-14T12:02:12.282706+0000 (oldest deadline 2023-07-14T12:02:38.182367+0000) 2023-07-14T12:02:42.436 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@osd.5.service' 2023-07-14T12:02:42.479 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T12:02:42.480 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-07-14T12:02:42.480 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-07-14T12:02:42.481 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@osd.6 2023-07-14T12:02:42.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:42 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:42.293+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:02:42.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:42 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:42.293+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.378517+0000 front 2023-07-14T12:02:07.378538+0000 (oldest deadline 2023-07-14T12:02:34.979250+0000) 2023-07-14T12:02:42.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:42 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:42.293+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:14.979892+0000 front 2023-07-14T12:02:14.979966+0000 (oldest deadline 2023-07-14T12:02:39.079887+0000) 2023-07-14T12:02:42.883 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:42 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:42.601+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:06.686677+0000 front 2023-07-14T12:02:06.686702+0000 (oldest deadline 2023-07-14T12:02:29.586617+0000) 2023-07-14T12:02:42.884 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:42 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:42.601+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:11.887528+0000 front 2023-07-14T12:02:11.887582+0000 (oldest deadline 2023-07-14T12:02:37.187740+0000) 2023-07-14T12:02:42.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:42 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:42.472+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:02:42.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:42 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:42.472+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:15.551211+0000 front 2023-07-14T12:02:15.551186+0000 (oldest deadline 2023-07-14T12:02:40.851263+0000) 2023-07-14T12:02:42.893 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:42 smithi017 systemd[1]: Stopping Ceph osd.6 for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T12:02:43.317 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:42 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:42.976+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.438626+0000 front 2023-07-14T12:02:07.438648+0000 (oldest deadline 2023-07-14T12:02:31.538779+0000) 2023-07-14T12:02:43.318 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:42 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:42.976+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:11.539352+0000 front 2023-07-14T12:02:11.539314+0000 (oldest deadline 2023-07-14T12:02:37.439216+0000) 2023-07-14T12:02:43.393 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:43 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:43.101+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:02:43.393 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:43 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:43.101+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:05.537048+0000 front 2023-07-14T12:02:05.536734+0000 (oldest deadline 2023-07-14T12:02:30.836845+0000) 2023-07-14T12:02:43.394 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:43 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:43.101+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:13.738547+0000 front 2023-07-14T12:02:13.738532+0000 (oldest deadline 2023-07-14T12:02:39.038483+0000) 2023-07-14T12:02:43.394 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:43 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:43.026+0000 7f7235268700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T12:02:43.394 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:43 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:43.026+0000 7f7235268700 -1 osd.6 96 *** Got signal Terminated *** 2023-07-14T12:02:43.394 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:43 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:43.026+0000 7f7235268700 -1 osd.6 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T12:02:43.395 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:43 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:43.030+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:57.178998+0000 front 2023-07-14T12:01:57.179119+0000 (oldest deadline 2023-07-14T12:02:21.279334+0000) 2023-07-14T12:02:43.395 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:43 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:43.030+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:08.182103+0000 front 2023-07-14T12:02:08.182077+0000 (oldest deadline 2023-07-14T12:02:32.281763+0000) 2023-07-14T12:02:43.395 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:43 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:43.030+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:12.282755+0000 front 2023-07-14T12:02:12.282706+0000 (oldest deadline 2023-07-14T12:02:38.182367+0000) 2023-07-14T12:02:43.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:43 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:43.338+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:02:43.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:43 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:43.338+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.378517+0000 front 2023-07-14T12:02:07.378538+0000 (oldest deadline 2023-07-14T12:02:34.979250+0000) 2023-07-14T12:02:43.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:43 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:43.338+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:14.979892+0000 front 2023-07-14T12:02:14.979966+0000 (oldest deadline 2023-07-14T12:02:39.079887+0000) 2023-07-14T12:02:43.883 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:43 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:43.635+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:06.686677+0000 front 2023-07-14T12:02:06.686702+0000 (oldest deadline 2023-07-14T12:02:29.586617+0000) 2023-07-14T12:02:43.884 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:43 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:43.635+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:11.887528+0000 front 2023-07-14T12:02:11.887582+0000 (oldest deadline 2023-07-14T12:02:37.187740+0000) 2023-07-14T12:02:43.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:43 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:43.474+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:02:43.885 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:43 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:43.474+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:15.551211+0000 front 2023-07-14T12:02:15.551186+0000 (oldest deadline 2023-07-14T12:02:40.851263+0000) 2023-07-14T12:02:44.270 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:44 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:44.010+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.438626+0000 front 2023-07-14T12:02:07.438648+0000 (oldest deadline 2023-07-14T12:02:31.538779+0000) 2023-07-14T12:02:44.271 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:44 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:44.010+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:11.539352+0000 front 2023-07-14T12:02:11.539314+0000 (oldest deadline 2023-07-14T12:02:37.439216+0000) 2023-07-14T12:02:44.393 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:44 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:44.074+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:02:44.394 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:44 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:44.074+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:05.537048+0000 front 2023-07-14T12:02:05.536734+0000 (oldest deadline 2023-07-14T12:02:30.836845+0000) 2023-07-14T12:02:44.394 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:44 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:44.074+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:13.738547+0000 front 2023-07-14T12:02:13.738532+0000 (oldest deadline 2023-07-14T12:02:39.038483+0000) 2023-07-14T12:02:44.394 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:44 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:44.079+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:57.178998+0000 front 2023-07-14T12:01:57.179119+0000 (oldest deadline 2023-07-14T12:02:21.279334+0000) 2023-07-14T12:02:44.394 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:44 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:44.079+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:08.182103+0000 front 2023-07-14T12:02:08.182077+0000 (oldest deadline 2023-07-14T12:02:32.281763+0000) 2023-07-14T12:02:44.395 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:44 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:44.079+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:12.282755+0000 front 2023-07-14T12:02:12.282706+0000 (oldest deadline 2023-07-14T12:02:38.182367+0000) 2023-07-14T12:02:44.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:44 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:44.291+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:02:44.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:44 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:44.291+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.378517+0000 front 2023-07-14T12:02:07.378538+0000 (oldest deadline 2023-07-14T12:02:34.979250+0000) 2023-07-14T12:02:44.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:44 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:44.291+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:14.979892+0000 front 2023-07-14T12:02:14.979966+0000 (oldest deadline 2023-07-14T12:02:39.079887+0000) 2023-07-14T12:02:44.883 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:44 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:44.476+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:02:44.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:44 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:44.476+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:15.551211+0000 front 2023-07-14T12:02:15.551186+0000 (oldest deadline 2023-07-14T12:02:40.851263+0000) 2023-07-14T12:02:44.884 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:44 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:44.608+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:06.686677+0000 front 2023-07-14T12:02:06.686702+0000 (oldest deadline 2023-07-14T12:02:29.586617+0000) 2023-07-14T12:02:44.885 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:44 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:44.608+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:11.887528+0000 front 2023-07-14T12:02:11.887582+0000 (oldest deadline 2023-07-14T12:02:37.187740+0000) 2023-07-14T12:02:45.318 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:45 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:45.026+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.438626+0000 front 2023-07-14T12:02:07.438648+0000 (oldest deadline 2023-07-14T12:02:31.538779+0000) 2023-07-14T12:02:45.319 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:45 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:45.026+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:11.539352+0000 front 2023-07-14T12:02:11.539314+0000 (oldest deadline 2023-07-14T12:02:37.439216+0000) 2023-07-14T12:02:45.393 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:45 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:45.083+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:57.178998+0000 front 2023-07-14T12:01:57.179119+0000 (oldest deadline 2023-07-14T12:02:21.279334+0000) 2023-07-14T12:02:45.394 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:45 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:45.083+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:08.182103+0000 front 2023-07-14T12:02:08.182077+0000 (oldest deadline 2023-07-14T12:02:32.281763+0000) 2023-07-14T12:02:45.394 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:45 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:45.083+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:12.282755+0000 front 2023-07-14T12:02:12.282706+0000 (oldest deadline 2023-07-14T12:02:38.182367+0000) 2023-07-14T12:02:45.394 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:45 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:45.029+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:02:45.395 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:45 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:45.029+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:05.537048+0000 front 2023-07-14T12:02:05.536734+0000 (oldest deadline 2023-07-14T12:02:30.836845+0000) 2023-07-14T12:02:45.395 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:45 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:45.029+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:13.738547+0000 front 2023-07-14T12:02:13.738532+0000 (oldest deadline 2023-07-14T12:02:39.038483+0000) 2023-07-14T12:02:45.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:45 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:45.339+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:02:45.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:45 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:45.339+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.378517+0000 front 2023-07-14T12:02:07.378538+0000 (oldest deadline 2023-07-14T12:02:34.979250+0000) 2023-07-14T12:02:45.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:45 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:45.339+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:14.979892+0000 front 2023-07-14T12:02:14.979966+0000 (oldest deadline 2023-07-14T12:02:39.079887+0000) 2023-07-14T12:02:45.883 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:45 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:45.613+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:06.686677+0000 front 2023-07-14T12:02:06.686702+0000 (oldest deadline 2023-07-14T12:02:29.586617+0000) 2023-07-14T12:02:45.884 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:45 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:45.613+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:11.887528+0000 front 2023-07-14T12:02:11.887582+0000 (oldest deadline 2023-07-14T12:02:37.187740+0000) 2023-07-14T12:02:45.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:45 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:45.431+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:02:45.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:45 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:45.431+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:15.551211+0000 front 2023-07-14T12:02:15.551186+0000 (oldest deadline 2023-07-14T12:02:40.851263+0000) 2023-07-14T12:02:46.340 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:46 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:46.049+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.438626+0000 front 2023-07-14T12:02:07.438648+0000 (oldest deadline 2023-07-14T12:02:31.538779+0000) 2023-07-14T12:02:46.341 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:46 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:46.049+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:11.539352+0000 front 2023-07-14T12:02:11.539314+0000 (oldest deadline 2023-07-14T12:02:37.439216+0000) 2023-07-14T12:02:46.393 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:45 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:45.982+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:02:46.393 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:45 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:45.982+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:05.537048+0000 front 2023-07-14T12:02:05.536734+0000 (oldest deadline 2023-07-14T12:02:30.836845+0000) 2023-07-14T12:02:46.394 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:45 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:45.982+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:13.738547+0000 front 2023-07-14T12:02:13.738532+0000 (oldest deadline 2023-07-14T12:02:39.038483+0000) 2023-07-14T12:02:46.394 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:46 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:46.037+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:57.178998+0000 front 2023-07-14T12:01:57.179119+0000 (oldest deadline 2023-07-14T12:02:21.279334+0000) 2023-07-14T12:02:46.394 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:46 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:46.037+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:08.182103+0000 front 2023-07-14T12:02:08.182077+0000 (oldest deadline 2023-07-14T12:02:32.281763+0000) 2023-07-14T12:02:46.395 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:46 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:46.037+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:12.282755+0000 front 2023-07-14T12:02:12.282706+0000 (oldest deadline 2023-07-14T12:02:38.182367+0000) 2023-07-14T12:02:46.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:46 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:46.360+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:02:46.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:46 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:46.360+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.378517+0000 front 2023-07-14T12:02:07.378538+0000 (oldest deadline 2023-07-14T12:02:34.979250+0000) 2023-07-14T12:02:46.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:46 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:46.360+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:14.979892+0000 front 2023-07-14T12:02:14.979966+0000 (oldest deadline 2023-07-14T12:02:39.079887+0000) 2023-07-14T12:02:46.883 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:46 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:46.645+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:06.686677+0000 front 2023-07-14T12:02:06.686702+0000 (oldest deadline 2023-07-14T12:02:29.586617+0000) 2023-07-14T12:02:46.884 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:46 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:46.645+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:11.887528+0000 front 2023-07-14T12:02:11.887582+0000 (oldest deadline 2023-07-14T12:02:37.187740+0000) 2023-07-14T12:02:46.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:46 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:46.415+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:02:46.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:46 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:46.415+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:15.551211+0000 front 2023-07-14T12:02:15.551186+0000 (oldest deadline 2023-07-14T12:02:40.851263+0000) 2023-07-14T12:02:47.324 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:47 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:47.079+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.438626+0000 front 2023-07-14T12:02:07.438648+0000 (oldest deadline 2023-07-14T12:02:31.538779+0000) 2023-07-14T12:02:47.325 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:47 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:47.079+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:11.539352+0000 front 2023-07-14T12:02:11.539314+0000 (oldest deadline 2023-07-14T12:02:37.439216+0000) 2023-07-14T12:02:47.393 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:47 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:47.009+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:02:47.393 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:47 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:47.009+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:05.537048+0000 front 2023-07-14T12:02:05.536734+0000 (oldest deadline 2023-07-14T12:02:30.836845+0000) 2023-07-14T12:02:47.393 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:47 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:47.009+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:13.738547+0000 front 2023-07-14T12:02:13.738532+0000 (oldest deadline 2023-07-14T12:02:39.038483+0000) 2023-07-14T12:02:47.394 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:47 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:47.009+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:24.339615+0000 front 2023-07-14T12:02:24.339656+0000 (oldest deadline 2023-07-14T12:02:46.039911+0000) 2023-07-14T12:02:47.394 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:47 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:47.022+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:57.178998+0000 front 2023-07-14T12:01:57.179119+0000 (oldest deadline 2023-07-14T12:02:21.279334+0000) 2023-07-14T12:02:47.394 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:47 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:47.022+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:08.182103+0000 front 2023-07-14T12:02:08.182077+0000 (oldest deadline 2023-07-14T12:02:32.281763+0000) 2023-07-14T12:02:47.394 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:47 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6[150358]: 2023-07-14T12:02:47.022+0000 7f723005b700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:12.282755+0000 front 2023-07-14T12:02:12.282706+0000 (oldest deadline 2023-07-14T12:02:38.182367+0000) 2023-07-14T12:02:47.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:47 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:47.345+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:02:47.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:47 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:47.345+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.378517+0000 front 2023-07-14T12:02:07.378538+0000 (oldest deadline 2023-07-14T12:02:34.979250+0000) 2023-07-14T12:02:47.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:47 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:47.345+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:14.979892+0000 front 2023-07-14T12:02:14.979966+0000 (oldest deadline 2023-07-14T12:02:39.079887+0000) 2023-07-14T12:02:47.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:47 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:47.345+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.081614+0000 front 2023-07-14T12:02:23.081691+0000 (oldest deadline 2023-07-14T12:02:47.181738+0000) 2023-07-14T12:02:47.627 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:47 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:47.396+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:02:47.627 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:47 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:47.396+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:15.551211+0000 front 2023-07-14T12:02:15.551186+0000 (oldest deadline 2023-07-14T12:02:40.851263+0000) 2023-07-14T12:02:47.627 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:47 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:47.396+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:21.852804+0000 front 2023-07-14T12:02:21.852675+0000 (oldest deadline 2023-07-14T12:02:46.552594+0000) 2023-07-14T12:02:47.883 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:47 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:47.647+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:06.686677+0000 front 2023-07-14T12:02:06.686702+0000 (oldest deadline 2023-07-14T12:02:29.586617+0000) 2023-07-14T12:02:47.884 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:47 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:47.647+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:11.887528+0000 front 2023-07-14T12:02:11.887582+0000 (oldest deadline 2023-07-14T12:02:37.187740+0000) 2023-07-14T12:02:47.884 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:47 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:47.647+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.089032+0000 front 2023-07-14T12:02:23.088900+0000 (oldest deadline 2023-07-14T12:02:47.188933+0000) 2023-07-14T12:02:48.358 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:48 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:48.076+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.438626+0000 front 2023-07-14T12:02:07.438648+0000 (oldest deadline 2023-07-14T12:02:31.538779+0000) 2023-07-14T12:02:48.358 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:48 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:48.076+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:11.539352+0000 front 2023-07-14T12:02:11.539314+0000 (oldest deadline 2023-07-14T12:02:37.439216+0000) 2023-07-14T12:02:48.358 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:48 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:48.076+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.240873+0000 front 2023-07-14T12:02:23.241004+0000 (oldest deadline 2023-07-14T12:02:47.940801+0000) 2023-07-14T12:02:48.393 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:47 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:47.981+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:02:48.393 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:47 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:47.981+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:05.537048+0000 front 2023-07-14T12:02:05.536734+0000 (oldest deadline 2023-07-14T12:02:30.836845+0000) 2023-07-14T12:02:48.393 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:47 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:47.981+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:13.738547+0000 front 2023-07-14T12:02:13.738532+0000 (oldest deadline 2023-07-14T12:02:39.038483+0000) 2023-07-14T12:02:48.394 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:47 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:47.981+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:24.339615+0000 front 2023-07-14T12:02:24.339656+0000 (oldest deadline 2023-07-14T12:02:46.039911+0000) 2023-07-14T12:02:48.604 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:48 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:48.346+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:02:48.604 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:48 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:48.346+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:15.551211+0000 front 2023-07-14T12:02:15.551186+0000 (oldest deadline 2023-07-14T12:02:40.851263+0000) 2023-07-14T12:02:48.605 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:48 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:48.346+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:21.852804+0000 front 2023-07-14T12:02:21.852675+0000 (oldest deadline 2023-07-14T12:02:46.552594+0000) 2023-07-14T12:02:48.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:48 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:48.378+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:02:48.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:48 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:48.378+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.378517+0000 front 2023-07-14T12:02:07.378538+0000 (oldest deadline 2023-07-14T12:02:34.979250+0000) 2023-07-14T12:02:48.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:48 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:48.378+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:14.979892+0000 front 2023-07-14T12:02:14.979966+0000 (oldest deadline 2023-07-14T12:02:39.079887+0000) 2023-07-14T12:02:48.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:48 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:48.378+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.081614+0000 front 2023-07-14T12:02:23.081691+0000 (oldest deadline 2023-07-14T12:02:47.181738+0000) 2023-07-14T12:02:48.883 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:48 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:48.624+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:06.686677+0000 front 2023-07-14T12:02:06.686702+0000 (oldest deadline 2023-07-14T12:02:29.586617+0000) 2023-07-14T12:02:48.884 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:48 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:48.624+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:11.887528+0000 front 2023-07-14T12:02:11.887582+0000 (oldest deadline 2023-07-14T12:02:37.187740+0000) 2023-07-14T12:02:48.884 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:48 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:48.624+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.089032+0000 front 2023-07-14T12:02:23.088900+0000 (oldest deadline 2023-07-14T12:02:47.188933+0000) 2023-07-14T12:02:48.963 INFO:journalctl@ceph.osd.6.smithi017.stdout:Jul 14 12:02:48 smithi017 bash[176518]: ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-6 2023-07-14T12:02:49.360 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:49 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:49.084+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.438626+0000 front 2023-07-14T12:02:07.438648+0000 (oldest deadline 2023-07-14T12:02:31.538779+0000) 2023-07-14T12:02:49.360 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:49 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:49.084+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:11.539352+0000 front 2023-07-14T12:02:11.539314+0000 (oldest deadline 2023-07-14T12:02:37.439216+0000) 2023-07-14T12:02:49.360 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:49 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:49.084+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.240873+0000 front 2023-07-14T12:02:23.241004+0000 (oldest deadline 2023-07-14T12:02:47.940801+0000) 2023-07-14T12:02:49.393 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:48 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:48.983+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:02:49.393 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:48 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:48.983+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:05.537048+0000 front 2023-07-14T12:02:05.536734+0000 (oldest deadline 2023-07-14T12:02:30.836845+0000) 2023-07-14T12:02:49.394 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:48 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:48.983+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:13.738547+0000 front 2023-07-14T12:02:13.738532+0000 (oldest deadline 2023-07-14T12:02:39.038483+0000) 2023-07-14T12:02:49.394 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:48 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:48.983+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:24.339615+0000 front 2023-07-14T12:02:24.339656+0000 (oldest deadline 2023-07-14T12:02:46.039911+0000) 2023-07-14T12:02:49.618 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:49 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:49.385+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:02:49.618 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:49 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:49.385+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:15.551211+0000 front 2023-07-14T12:02:15.551186+0000 (oldest deadline 2023-07-14T12:02:40.851263+0000) 2023-07-14T12:02:49.619 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:49 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:49.385+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:21.852804+0000 front 2023-07-14T12:02:21.852675+0000 (oldest deadline 2023-07-14T12:02:46.552594+0000) 2023-07-14T12:02:49.860 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:49 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:49.394+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:02:49.860 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:49 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:49.394+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.378517+0000 front 2023-07-14T12:02:07.378538+0000 (oldest deadline 2023-07-14T12:02:34.979250+0000) 2023-07-14T12:02:49.860 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:49 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:49.394+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:14.979892+0000 front 2023-07-14T12:02:14.979966+0000 (oldest deadline 2023-07-14T12:02:39.079887+0000) 2023-07-14T12:02:49.861 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:49 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:49.394+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.081614+0000 front 2023-07-14T12:02:23.081691+0000 (oldest deadline 2023-07-14T12:02:47.181738+0000) 2023-07-14T12:02:49.883 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:49 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:49.638+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:06.686677+0000 front 2023-07-14T12:02:06.686702+0000 (oldest deadline 2023-07-14T12:02:29.586617+0000) 2023-07-14T12:02:49.884 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:49 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:49.638+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:11.887528+0000 front 2023-07-14T12:02:11.887582+0000 (oldest deadline 2023-07-14T12:02:37.187740+0000) 2023-07-14T12:02:49.884 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:49 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:49.638+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.089032+0000 front 2023-07-14T12:02:23.088900+0000 (oldest deadline 2023-07-14T12:02:47.188933+0000) 2023-07-14T12:02:50.360 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:50 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:50.103+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.438626+0000 front 2023-07-14T12:02:07.438648+0000 (oldest deadline 2023-07-14T12:02:31.538779+0000) 2023-07-14T12:02:50.360 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:50 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:50.103+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:11.539352+0000 front 2023-07-14T12:02:11.539314+0000 (oldest deadline 2023-07-14T12:02:37.439216+0000) 2023-07-14T12:02:50.361 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:50 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:50.103+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.240873+0000 front 2023-07-14T12:02:23.241004+0000 (oldest deadline 2023-07-14T12:02:47.940801+0000) 2023-07-14T12:02:50.393 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:49 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:49.986+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:02:50.393 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:49 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:49.986+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:05.537048+0000 front 2023-07-14T12:02:05.536734+0000 (oldest deadline 2023-07-14T12:02:30.836845+0000) 2023-07-14T12:02:50.394 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:49 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:49.986+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:13.738547+0000 front 2023-07-14T12:02:13.738532+0000 (oldest deadline 2023-07-14T12:02:39.038483+0000) 2023-07-14T12:02:50.394 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:49 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:49.986+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:24.339615+0000 front 2023-07-14T12:02:24.339656+0000 (oldest deadline 2023-07-14T12:02:46.039911+0000) 2023-07-14T12:02:50.633 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:50 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:50.590+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:06.686677+0000 front 2023-07-14T12:02:06.686702+0000 (oldest deadline 2023-07-14T12:02:29.586617+0000) 2023-07-14T12:02:50.634 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:50 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:50.590+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:11.887528+0000 front 2023-07-14T12:02:11.887582+0000 (oldest deadline 2023-07-14T12:02:37.187740+0000) 2023-07-14T12:02:50.634 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:50 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:50.590+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.089032+0000 front 2023-07-14T12:02:23.088900+0000 (oldest deadline 2023-07-14T12:02:47.188933+0000) 2023-07-14T12:02:50.634 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:50 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:50.392+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:02:50.635 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:50 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:50.392+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:15.551211+0000 front 2023-07-14T12:02:15.551186+0000 (oldest deadline 2023-07-14T12:02:40.851263+0000) 2023-07-14T12:02:50.635 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:50 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:50.392+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:21.852804+0000 front 2023-07-14T12:02:21.852675+0000 (oldest deadline 2023-07-14T12:02:46.552594+0000) 2023-07-14T12:02:50.860 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:50 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:50.403+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:02:50.861 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:50 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:50.403+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.378517+0000 front 2023-07-14T12:02:07.378538+0000 (oldest deadline 2023-07-14T12:02:34.979250+0000) 2023-07-14T12:02:50.861 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:50 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:50.403+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:14.979892+0000 front 2023-07-14T12:02:14.979966+0000 (oldest deadline 2023-07-14T12:02:39.079887+0000) 2023-07-14T12:02:50.861 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:50 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:50.403+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.081614+0000 front 2023-07-14T12:02:23.081691+0000 (oldest deadline 2023-07-14T12:02:47.181738+0000) 2023-07-14T12:02:51.073 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:50 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:50.949+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:02:51.073 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:50 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:50.949+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:05.537048+0000 front 2023-07-14T12:02:05.536734+0000 (oldest deadline 2023-07-14T12:02:30.836845+0000) 2023-07-14T12:02:51.074 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:50 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:50.949+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:13.738547+0000 front 2023-07-14T12:02:13.738532+0000 (oldest deadline 2023-07-14T12:02:39.038483+0000) 2023-07-14T12:02:51.074 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:50 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:50.949+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:24.339615+0000 front 2023-07-14T12:02:24.339656+0000 (oldest deadline 2023-07-14T12:02:46.039911+0000) 2023-07-14T12:02:51.100 DEBUG:teuthology.orchestra.run.smithi017:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@osd.6.service' 2023-07-14T12:02:51.192 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T12:02:51.192 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-07-14T12:02:51.193 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-07-14T12:02:51.193 DEBUG:teuthology.orchestra.run.smithi093:> sudo systemctl stop ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@osd.7 2023-07-14T12:02:51.360 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:51 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:51.368+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:02:51.360 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:51 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:51.368+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.378517+0000 front 2023-07-14T12:02:07.378538+0000 (oldest deadline 2023-07-14T12:02:34.979250+0000) 2023-07-14T12:02:51.360 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:51 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:51.368+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:14.979892+0000 front 2023-07-14T12:02:14.979966+0000 (oldest deadline 2023-07-14T12:02:39.079887+0000) 2023-07-14T12:02:51.361 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:51 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:51.368+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.081614+0000 front 2023-07-14T12:02:23.081691+0000 (oldest deadline 2023-07-14T12:02:47.181738+0000) 2023-07-14T12:02:51.361 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:51 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:51.119+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.438626+0000 front 2023-07-14T12:02:07.438648+0000 (oldest deadline 2023-07-14T12:02:31.538779+0000) 2023-07-14T12:02:51.361 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:51 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:51.119+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:11.539352+0000 front 2023-07-14T12:02:11.539314+0000 (oldest deadline 2023-07-14T12:02:37.439216+0000) 2023-07-14T12:02:51.361 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:51 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:51.119+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.240873+0000 front 2023-07-14T12:02:23.241004+0000 (oldest deadline 2023-07-14T12:02:47.940801+0000) 2023-07-14T12:02:51.362 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:51 smithi093 systemd[1]: Stopping Ceph osd.7 for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T12:02:51.860 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:51 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:51.583+0000 7f2fb3968700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T12:02:51.860 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:51 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:51.583+0000 7f2fb3968700 -1 osd.7 96 *** Got signal Terminated *** 2023-07-14T12:02:51.861 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:51 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:51.583+0000 7f2fb3968700 -1 osd.7 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T12:02:51.883 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:51 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:51.613+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:06.686677+0000 front 2023-07-14T12:02:06.686702+0000 (oldest deadline 2023-07-14T12:02:29.586617+0000) 2023-07-14T12:02:51.884 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:51 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:51.613+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:11.887528+0000 front 2023-07-14T12:02:11.887582+0000 (oldest deadline 2023-07-14T12:02:37.187740+0000) 2023-07-14T12:02:51.884 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:51 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:51.613+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.089032+0000 front 2023-07-14T12:02:23.088900+0000 (oldest deadline 2023-07-14T12:02:47.188933+0000) 2023-07-14T12:02:51.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:51 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:51.412+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:02:51.885 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:51 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:51.412+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:15.551211+0000 front 2023-07-14T12:02:15.551186+0000 (oldest deadline 2023-07-14T12:02:40.851263+0000) 2023-07-14T12:02:51.885 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:51 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:51.412+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:21.852804+0000 front 2023-07-14T12:02:21.852675+0000 (oldest deadline 2023-07-14T12:02:46.552594+0000) 2023-07-14T12:02:52.360 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:52 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:52.100+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.438626+0000 front 2023-07-14T12:02:07.438648+0000 (oldest deadline 2023-07-14T12:02:31.538779+0000) 2023-07-14T12:02:52.360 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:52 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:52.100+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:11.539352+0000 front 2023-07-14T12:02:11.539314+0000 (oldest deadline 2023-07-14T12:02:37.439216+0000) 2023-07-14T12:02:52.360 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:52 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:52.100+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.240873+0000 front 2023-07-14T12:02:23.241004+0000 (oldest deadline 2023-07-14T12:02:47.940801+0000) 2023-07-14T12:02:52.393 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:51 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:51.924+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:02:52.394 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:51 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:51.924+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:05.537048+0000 front 2023-07-14T12:02:05.536734+0000 (oldest deadline 2023-07-14T12:02:30.836845+0000) 2023-07-14T12:02:52.394 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:51 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:51.924+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:13.738547+0000 front 2023-07-14T12:02:13.738532+0000 (oldest deadline 2023-07-14T12:02:39.038483+0000) 2023-07-14T12:02:52.394 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:51 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:51.924+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:24.339615+0000 front 2023-07-14T12:02:24.339656+0000 (oldest deadline 2023-07-14T12:02:46.039911+0000) 2023-07-14T12:02:52.860 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:52 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:52.398+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:02:52.860 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:52 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:52.398+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.378517+0000 front 2023-07-14T12:02:07.378538+0000 (oldest deadline 2023-07-14T12:02:34.979250+0000) 2023-07-14T12:02:52.860 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:52 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:52.398+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:14.979892+0000 front 2023-07-14T12:02:14.979966+0000 (oldest deadline 2023-07-14T12:02:39.079887+0000) 2023-07-14T12:02:52.860 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:52 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:52.398+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.081614+0000 front 2023-07-14T12:02:23.081691+0000 (oldest deadline 2023-07-14T12:02:47.181738+0000) 2023-07-14T12:02:52.883 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:52 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:52.440+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:02:52.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:52 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:52.440+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:15.551211+0000 front 2023-07-14T12:02:15.551186+0000 (oldest deadline 2023-07-14T12:02:40.851263+0000) 2023-07-14T12:02:52.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:52 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:52.440+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:21.852804+0000 front 2023-07-14T12:02:21.852675+0000 (oldest deadline 2023-07-14T12:02:46.552594+0000) 2023-07-14T12:02:52.884 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:52 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:52.646+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:06.686677+0000 front 2023-07-14T12:02:06.686702+0000 (oldest deadline 2023-07-14T12:02:29.586617+0000) 2023-07-14T12:02:52.885 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:52 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:52.646+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:11.887528+0000 front 2023-07-14T12:02:11.887582+0000 (oldest deadline 2023-07-14T12:02:37.187740+0000) 2023-07-14T12:02:52.885 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:52 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:52.646+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.089032+0000 front 2023-07-14T12:02:23.088900+0000 (oldest deadline 2023-07-14T12:02:47.188933+0000) 2023-07-14T12:02:53.360 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:53 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:53.072+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.438626+0000 front 2023-07-14T12:02:07.438648+0000 (oldest deadline 2023-07-14T12:02:31.538779+0000) 2023-07-14T12:02:53.360 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:53 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:53.072+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:11.539352+0000 front 2023-07-14T12:02:11.539314+0000 (oldest deadline 2023-07-14T12:02:37.439216+0000) 2023-07-14T12:02:53.360 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:53 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:53.072+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.240873+0000 front 2023-07-14T12:02:23.241004+0000 (oldest deadline 2023-07-14T12:02:47.940801+0000) 2023-07-14T12:02:53.393 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:52 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:52.964+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:02:53.393 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:52 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:52.964+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:05.537048+0000 front 2023-07-14T12:02:05.536734+0000 (oldest deadline 2023-07-14T12:02:30.836845+0000) 2023-07-14T12:02:53.394 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:52 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:52.964+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:13.738547+0000 front 2023-07-14T12:02:13.738532+0000 (oldest deadline 2023-07-14T12:02:39.038483+0000) 2023-07-14T12:02:53.394 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:52 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:52.964+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:24.339615+0000 front 2023-07-14T12:02:24.339656+0000 (oldest deadline 2023-07-14T12:02:46.039911+0000) 2023-07-14T12:02:53.633 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:53 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:53.402+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:02:53.634 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:53 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:53.402+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:15.551211+0000 front 2023-07-14T12:02:15.551186+0000 (oldest deadline 2023-07-14T12:02:40.851263+0000) 2023-07-14T12:02:53.634 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:53 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:53.402+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:21.852804+0000 front 2023-07-14T12:02:21.852675+0000 (oldest deadline 2023-07-14T12:02:46.552594+0000) 2023-07-14T12:02:53.860 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:53 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:53.405+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:02:53.860 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:53 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:53.405+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.378517+0000 front 2023-07-14T12:02:07.378538+0000 (oldest deadline 2023-07-14T12:02:34.979250+0000) 2023-07-14T12:02:53.860 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:53 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:53.405+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:14.979892+0000 front 2023-07-14T12:02:14.979966+0000 (oldest deadline 2023-07-14T12:02:39.079887+0000) 2023-07-14T12:02:53.861 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:53 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:53.405+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.081614+0000 front 2023-07-14T12:02:23.081691+0000 (oldest deadline 2023-07-14T12:02:47.181738+0000) 2023-07-14T12:02:54.133 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:53 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:53.690+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:06.686677+0000 front 2023-07-14T12:02:06.686702+0000 (oldest deadline 2023-07-14T12:02:29.586617+0000) 2023-07-14T12:02:54.134 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:53 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:53.690+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:11.887528+0000 front 2023-07-14T12:02:11.887582+0000 (oldest deadline 2023-07-14T12:02:37.187740+0000) 2023-07-14T12:02:54.134 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:53 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:53.690+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.089032+0000 front 2023-07-14T12:02:23.088900+0000 (oldest deadline 2023-07-14T12:02:47.188933+0000) 2023-07-14T12:02:54.360 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:54 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:54.096+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.438626+0000 front 2023-07-14T12:02:07.438648+0000 (oldest deadline 2023-07-14T12:02:31.538779+0000) 2023-07-14T12:02:54.360 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:54 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:54.096+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:11.539352+0000 front 2023-07-14T12:02:11.539314+0000 (oldest deadline 2023-07-14T12:02:37.439216+0000) 2023-07-14T12:02:54.360 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:54 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:54.096+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.240873+0000 front 2023-07-14T12:02:23.241004+0000 (oldest deadline 2023-07-14T12:02:47.940801+0000) 2023-07-14T12:02:54.360 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:54 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:54.096+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:27.941210+0000 front 2023-07-14T12:02:27.941245+0000 (oldest deadline 2023-07-14T12:02:53.241417+0000) 2023-07-14T12:02:54.393 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:53 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:53.956+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:02:54.393 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:53 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:53.956+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:05.537048+0000 front 2023-07-14T12:02:05.536734+0000 (oldest deadline 2023-07-14T12:02:30.836845+0000) 2023-07-14T12:02:54.394 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:53 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:53.956+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:13.738547+0000 front 2023-07-14T12:02:13.738532+0000 (oldest deadline 2023-07-14T12:02:39.038483+0000) 2023-07-14T12:02:54.394 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:53 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:53.956+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:24.339615+0000 front 2023-07-14T12:02:24.339656+0000 (oldest deadline 2023-07-14T12:02:46.039911+0000) 2023-07-14T12:02:54.692 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:54 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:54.406+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:02:54.692 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:54 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:54.406+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:15.551211+0000 front 2023-07-14T12:02:15.551186+0000 (oldest deadline 2023-07-14T12:02:40.851263+0000) 2023-07-14T12:02:54.692 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:54 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:54.406+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:21.852804+0000 front 2023-07-14T12:02:21.852675+0000 (oldest deadline 2023-07-14T12:02:46.552594+0000) 2023-07-14T12:02:54.860 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:54 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:54.384+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:02:54.860 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:54 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:54.384+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.378517+0000 front 2023-07-14T12:02:07.378538+0000 (oldest deadline 2023-07-14T12:02:34.979250+0000) 2023-07-14T12:02:54.860 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:54 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:54.384+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:14.979892+0000 front 2023-07-14T12:02:14.979966+0000 (oldest deadline 2023-07-14T12:02:39.079887+0000) 2023-07-14T12:02:54.861 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:54 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:54.384+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.081614+0000 front 2023-07-14T12:02:23.081691+0000 (oldest deadline 2023-07-14T12:02:47.181738+0000) 2023-07-14T12:02:54.861 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:54 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:54.384+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:27.182181+0000 front 2023-07-14T12:02:27.182198+0000 (oldest deadline 2023-07-14T12:02:54.182946+0000) 2023-07-14T12:02:55.133 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:54 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:54.712+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:06.686677+0000 front 2023-07-14T12:02:06.686702+0000 (oldest deadline 2023-07-14T12:02:29.586617+0000) 2023-07-14T12:02:55.133 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:54 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:54.712+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:11.887528+0000 front 2023-07-14T12:02:11.887582+0000 (oldest deadline 2023-07-14T12:02:37.187740+0000) 2023-07-14T12:02:55.134 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:54 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:54.712+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.089032+0000 front 2023-07-14T12:02:23.088900+0000 (oldest deadline 2023-07-14T12:02:47.188933+0000) 2023-07-14T12:02:55.360 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:55 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:55.055+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.438626+0000 front 2023-07-14T12:02:07.438648+0000 (oldest deadline 2023-07-14T12:02:31.538779+0000) 2023-07-14T12:02:55.360 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:55 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:55.055+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:11.539352+0000 front 2023-07-14T12:02:11.539314+0000 (oldest deadline 2023-07-14T12:02:37.439216+0000) 2023-07-14T12:02:55.360 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:55 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:55.055+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.240873+0000 front 2023-07-14T12:02:23.241004+0000 (oldest deadline 2023-07-14T12:02:47.940801+0000) 2023-07-14T12:02:55.361 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:55 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:55.055+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:27.941210+0000 front 2023-07-14T12:02:27.941245+0000 (oldest deadline 2023-07-14T12:02:53.241417+0000) 2023-07-14T12:02:55.393 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:54 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:54.930+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:02:55.393 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:54 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:54.930+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:05.537048+0000 front 2023-07-14T12:02:05.536734+0000 (oldest deadline 2023-07-14T12:02:30.836845+0000) 2023-07-14T12:02:55.393 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:54 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:54.930+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:13.738547+0000 front 2023-07-14T12:02:13.738532+0000 (oldest deadline 2023-07-14T12:02:39.038483+0000) 2023-07-14T12:02:55.394 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:54 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:54.930+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:24.339615+0000 front 2023-07-14T12:02:24.339656+0000 (oldest deadline 2023-07-14T12:02:46.039911+0000) 2023-07-14T12:02:55.689 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:55 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:55.448+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:02:55.690 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:55 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:55.448+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:15.551211+0000 front 2023-07-14T12:02:15.551186+0000 (oldest deadline 2023-07-14T12:02:40.851263+0000) 2023-07-14T12:02:55.690 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:55 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:55.448+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:21.852804+0000 front 2023-07-14T12:02:21.852675+0000 (oldest deadline 2023-07-14T12:02:46.552594+0000) 2023-07-14T12:02:55.860 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:55 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:55.408+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:02:55.860 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:55 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:55.408+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.378517+0000 front 2023-07-14T12:02:07.378538+0000 (oldest deadline 2023-07-14T12:02:34.979250+0000) 2023-07-14T12:02:55.860 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:55 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:55.408+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:14.979892+0000 front 2023-07-14T12:02:14.979966+0000 (oldest deadline 2023-07-14T12:02:39.079887+0000) 2023-07-14T12:02:55.861 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:55 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:55.408+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.081614+0000 front 2023-07-14T12:02:23.081691+0000 (oldest deadline 2023-07-14T12:02:47.181738+0000) 2023-07-14T12:02:55.861 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:55 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:55.408+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:27.182181+0000 front 2023-07-14T12:02:27.182198+0000 (oldest deadline 2023-07-14T12:02:54.182946+0000) 2023-07-14T12:02:56.133 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:55 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:55.710+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:06.686677+0000 front 2023-07-14T12:02:06.686702+0000 (oldest deadline 2023-07-14T12:02:29.586617+0000) 2023-07-14T12:02:56.134 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:55 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:55.710+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:11.887528+0000 front 2023-07-14T12:02:11.887582+0000 (oldest deadline 2023-07-14T12:02:37.187740+0000) 2023-07-14T12:02:56.134 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:55 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:55.710+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.089032+0000 front 2023-07-14T12:02:23.088900+0000 (oldest deadline 2023-07-14T12:02:47.188933+0000) 2023-07-14T12:02:56.134 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:55 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:55.710+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.290100+0000 front 2023-07-14T12:02:31.290230+0000 (oldest deadline 2023-07-14T12:02:55.390035+0000) 2023-07-14T12:02:56.143 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:55 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:55.881+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:02:56.144 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:55 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:55.881+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:05.537048+0000 front 2023-07-14T12:02:05.536734+0000 (oldest deadline 2023-07-14T12:02:30.836845+0000) 2023-07-14T12:02:56.144 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:55 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:55.881+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:13.738547+0000 front 2023-07-14T12:02:13.738532+0000 (oldest deadline 2023-07-14T12:02:39.038483+0000) 2023-07-14T12:02:56.144 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:55 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:55.881+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:24.339615+0000 front 2023-07-14T12:02:24.339656+0000 (oldest deadline 2023-07-14T12:02:46.039911+0000) 2023-07-14T12:02:56.360 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:56 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:56.008+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.438626+0000 front 2023-07-14T12:02:07.438648+0000 (oldest deadline 2023-07-14T12:02:31.538779+0000) 2023-07-14T12:02:56.360 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:56 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:56.008+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:11.539352+0000 front 2023-07-14T12:02:11.539314+0000 (oldest deadline 2023-07-14T12:02:37.439216+0000) 2023-07-14T12:02:56.361 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:56 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:56.008+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.240873+0000 front 2023-07-14T12:02:23.241004+0000 (oldest deadline 2023-07-14T12:02:47.940801+0000) 2023-07-14T12:02:56.361 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:56 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7[145070]: 2023-07-14T12:02:56.008+0000 7f2faff5e700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:27.941210+0000 front 2023-07-14T12:02:27.941245+0000 (oldest deadline 2023-07-14T12:02:53.241417+0000) 2023-07-14T12:02:56.860 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:56 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:56.405+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:02:56.860 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:56 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:56.405+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.378517+0000 front 2023-07-14T12:02:07.378538+0000 (oldest deadline 2023-07-14T12:02:34.979250+0000) 2023-07-14T12:02:56.860 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:56 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:56.405+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:14.979892+0000 front 2023-07-14T12:02:14.979966+0000 (oldest deadline 2023-07-14T12:02:39.079887+0000) 2023-07-14T12:02:56.861 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:56 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:56.405+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.081614+0000 front 2023-07-14T12:02:23.081691+0000 (oldest deadline 2023-07-14T12:02:47.181738+0000) 2023-07-14T12:02:56.861 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:56 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:56.405+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:27.182181+0000 front 2023-07-14T12:02:27.182198+0000 (oldest deadline 2023-07-14T12:02:54.182946+0000) 2023-07-14T12:02:56.883 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:56 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:56.440+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:02:56.883 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:56 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:56.440+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:15.551211+0000 front 2023-07-14T12:02:15.551186+0000 (oldest deadline 2023-07-14T12:02:40.851263+0000) 2023-07-14T12:02:56.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:56 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:56.440+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:21.852804+0000 front 2023-07-14T12:02:21.852675+0000 (oldest deadline 2023-07-14T12:02:46.552594+0000) 2023-07-14T12:02:56.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:56 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:56.440+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.253639+0000 front 2023-07-14T12:02:31.253703+0000 (oldest deadline 2023-07-14T12:02:55.953589+0000) 2023-07-14T12:02:56.884 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:56 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:56.687+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:06.686677+0000 front 2023-07-14T12:02:06.686702+0000 (oldest deadline 2023-07-14T12:02:29.586617+0000) 2023-07-14T12:02:56.884 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:56 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:56.687+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:11.887528+0000 front 2023-07-14T12:02:11.887582+0000 (oldest deadline 2023-07-14T12:02:37.187740+0000) 2023-07-14T12:02:56.885 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:56 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:56.687+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.089032+0000 front 2023-07-14T12:02:23.088900+0000 (oldest deadline 2023-07-14T12:02:47.188933+0000) 2023-07-14T12:02:56.885 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:56 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:56.687+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.290100+0000 front 2023-07-14T12:02:31.290230+0000 (oldest deadline 2023-07-14T12:02:55.390035+0000) 2023-07-14T12:02:57.143 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:56 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:56.904+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:02:57.143 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:56 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:56.904+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:05.537048+0000 front 2023-07-14T12:02:05.536734+0000 (oldest deadline 2023-07-14T12:02:30.836845+0000) 2023-07-14T12:02:57.143 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:56 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:56.904+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:13.738547+0000 front 2023-07-14T12:02:13.738532+0000 (oldest deadline 2023-07-14T12:02:39.038483+0000) 2023-07-14T12:02:57.144 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:56 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:56.904+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:24.339615+0000 front 2023-07-14T12:02:24.339656+0000 (oldest deadline 2023-07-14T12:02:46.039911+0000) 2023-07-14T12:02:57.144 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:56 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:56.904+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.841753+0000 front 2023-07-14T12:02:31.841888+0000 (oldest deadline 2023-07-14T12:02:55.941910+0000) 2023-07-14T12:02:57.431 INFO:journalctl@ceph.osd.7.smithi093.stdout:Jul 14 12:02:57 smithi093 bash[157267]: ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-7 2023-07-14T12:02:57.432 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:57 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:57.428+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:02:57.432 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:57 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:57.428+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.378517+0000 front 2023-07-14T12:02:07.378538+0000 (oldest deadline 2023-07-14T12:02:34.979250+0000) 2023-07-14T12:02:57.432 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:57 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:57.428+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:14.979892+0000 front 2023-07-14T12:02:14.979966+0000 (oldest deadline 2023-07-14T12:02:39.079887+0000) 2023-07-14T12:02:57.433 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:57 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:57.428+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.081614+0000 front 2023-07-14T12:02:23.081691+0000 (oldest deadline 2023-07-14T12:02:47.181738+0000) 2023-07-14T12:02:57.433 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:57 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:57.428+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:27.182181+0000 front 2023-07-14T12:02:27.182198+0000 (oldest deadline 2023-07-14T12:02:54.182946+0000) 2023-07-14T12:02:57.886 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:57 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:57.688+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:06.686677+0000 front 2023-07-14T12:02:06.686702+0000 (oldest deadline 2023-07-14T12:02:29.586617+0000) 2023-07-14T12:02:58.178 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:57 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:57.688+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:11.887528+0000 front 2023-07-14T12:02:11.887582+0000 (oldest deadline 2023-07-14T12:02:37.187740+0000) 2023-07-14T12:02:58.178 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:57 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:57.688+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.089032+0000 front 2023-07-14T12:02:23.088900+0000 (oldest deadline 2023-07-14T12:02:47.188933+0000) 2023-07-14T12:02:58.178 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:57 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:57.688+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.290100+0000 front 2023-07-14T12:02:31.290230+0000 (oldest deadline 2023-07-14T12:02:55.390035+0000) 2023-07-14T12:02:58.179 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:57 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:57.476+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:02:58.179 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:57 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:57.476+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:15.551211+0000 front 2023-07-14T12:02:15.551186+0000 (oldest deadline 2023-07-14T12:02:40.851263+0000) 2023-07-14T12:02:58.179 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:57 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:57.476+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:21.852804+0000 front 2023-07-14T12:02:21.852675+0000 (oldest deadline 2023-07-14T12:02:46.552594+0000) 2023-07-14T12:02:58.179 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:57 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:57.476+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.253639+0000 front 2023-07-14T12:02:31.253703+0000 (oldest deadline 2023-07-14T12:02:55.953589+0000) 2023-07-14T12:02:58.180 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:57 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:57.870+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:02:58.180 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:57 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:57.870+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:05.537048+0000 front 2023-07-14T12:02:05.536734+0000 (oldest deadline 2023-07-14T12:02:30.836845+0000) 2023-07-14T12:02:58.181 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:57 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:57.870+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:13.738547+0000 front 2023-07-14T12:02:13.738532+0000 (oldest deadline 2023-07-14T12:02:39.038483+0000) 2023-07-14T12:02:58.181 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:57 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:57.870+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:24.339615+0000 front 2023-07-14T12:02:24.339656+0000 (oldest deadline 2023-07-14T12:02:46.039911+0000) 2023-07-14T12:02:58.181 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:57 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:57.870+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.841753+0000 front 2023-07-14T12:02:31.841888+0000 (oldest deadline 2023-07-14T12:02:55.941910+0000) 2023-07-14T12:02:58.688 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:58 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:58.426+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:02:58.689 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:58 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:58.426+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.378517+0000 front 2023-07-14T12:02:07.378538+0000 (oldest deadline 2023-07-14T12:02:34.979250+0000) 2023-07-14T12:02:58.689 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:58 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:58.426+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:14.979892+0000 front 2023-07-14T12:02:14.979966+0000 (oldest deadline 2023-07-14T12:02:39.079887+0000) 2023-07-14T12:02:58.689 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:58 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:58.426+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.081614+0000 front 2023-07-14T12:02:23.081691+0000 (oldest deadline 2023-07-14T12:02:47.181738+0000) 2023-07-14T12:02:58.689 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:58 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:58.426+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:27.182181+0000 front 2023-07-14T12:02:27.182198+0000 (oldest deadline 2023-07-14T12:02:54.182946+0000) 2023-07-14T12:02:58.694 DEBUG:teuthology.orchestra.run.smithi093:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@osd.7.service' 2023-07-14T12:02:58.733 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T12:02:58.734 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-07-14T12:02:58.735 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-07-14T12:02:58.735 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@osd.8 2023-07-14T12:02:58.765 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:58 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:58.440+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:02:58.766 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:58 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:58.440+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:15.551211+0000 front 2023-07-14T12:02:15.551186+0000 (oldest deadline 2023-07-14T12:02:40.851263+0000) 2023-07-14T12:02:58.766 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:58 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:58.440+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:21.852804+0000 front 2023-07-14T12:02:21.852675+0000 (oldest deadline 2023-07-14T12:02:46.552594+0000) 2023-07-14T12:02:58.766 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:58 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:58.440+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.253639+0000 front 2023-07-14T12:02:31.253703+0000 (oldest deadline 2023-07-14T12:02:55.953589+0000) 2023-07-14T12:02:58.766 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:58 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:58.665+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:06.686677+0000 front 2023-07-14T12:02:06.686702+0000 (oldest deadline 2023-07-14T12:02:29.586617+0000) 2023-07-14T12:02:58.767 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:58 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:58.665+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:11.887528+0000 front 2023-07-14T12:02:11.887582+0000 (oldest deadline 2023-07-14T12:02:37.187740+0000) 2023-07-14T12:02:58.767 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:58 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:58.665+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.089032+0000 front 2023-07-14T12:02:23.088900+0000 (oldest deadline 2023-07-14T12:02:47.188933+0000) 2023-07-14T12:02:58.767 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:58 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:58.665+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.290100+0000 front 2023-07-14T12:02:31.290230+0000 (oldest deadline 2023-07-14T12:02:55.390035+0000) 2023-07-14T12:02:59.098 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:58 smithi097 systemd[1]: Stopping Ceph osd.8 for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T12:02:59.143 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:58 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:58.846+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:02:59.143 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:58 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:58.846+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:05.537048+0000 front 2023-07-14T12:02:05.536734+0000 (oldest deadline 2023-07-14T12:02:30.836845+0000) 2023-07-14T12:02:59.144 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:58 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:58.846+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:13.738547+0000 front 2023-07-14T12:02:13.738532+0000 (oldest deadline 2023-07-14T12:02:39.038483+0000) 2023-07-14T12:02:59.144 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:58 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:58.846+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:24.339615+0000 front 2023-07-14T12:02:24.339656+0000 (oldest deadline 2023-07-14T12:02:46.039911+0000) 2023-07-14T12:02:59.144 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:58 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:58.846+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.841753+0000 front 2023-07-14T12:02:31.841888+0000 (oldest deadline 2023-07-14T12:02:55.941910+0000) 2023-07-14T12:02:59.375 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:59 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:59.118+0000 7ff866a84700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T12:02:59.375 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:59 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:59.118+0000 7ff866a84700 -1 osd.8 96 *** Got signal Terminated *** 2023-07-14T12:02:59.376 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:59 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:59.118+0000 7ff866a84700 -1 osd.8 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T12:02:59.633 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:59 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:59.629+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:06.686677+0000 front 2023-07-14T12:02:06.686702+0000 (oldest deadline 2023-07-14T12:02:29.586617+0000) 2023-07-14T12:02:59.634 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:59 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:59.629+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:11.887528+0000 front 2023-07-14T12:02:11.887582+0000 (oldest deadline 2023-07-14T12:02:37.187740+0000) 2023-07-14T12:02:59.634 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:59 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:59.629+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.089032+0000 front 2023-07-14T12:02:23.088900+0000 (oldest deadline 2023-07-14T12:02:47.188933+0000) 2023-07-14T12:02:59.634 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:02:59 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:02:59.629+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.290100+0000 front 2023-07-14T12:02:31.290230+0000 (oldest deadline 2023-07-14T12:02:55.390035+0000) 2023-07-14T12:02:59.634 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:59 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:59.395+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:02:59.635 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:59 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:59.395+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:15.551211+0000 front 2023-07-14T12:02:15.551186+0000 (oldest deadline 2023-07-14T12:02:40.851263+0000) 2023-07-14T12:02:59.635 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:59 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:59.395+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:21.852804+0000 front 2023-07-14T12:02:21.852675+0000 (oldest deadline 2023-07-14T12:02:46.552594+0000) 2023-07-14T12:02:59.635 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:02:59 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:02:59.395+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.253639+0000 front 2023-07-14T12:02:31.253703+0000 (oldest deadline 2023-07-14T12:02:55.953589+0000) 2023-07-14T12:02:59.860 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:59 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:59.453+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:02:59.861 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:59 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:59.453+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.378517+0000 front 2023-07-14T12:02:07.378538+0000 (oldest deadline 2023-07-14T12:02:34.979250+0000) 2023-07-14T12:02:59.861 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:59 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:59.453+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:14.979892+0000 front 2023-07-14T12:02:14.979966+0000 (oldest deadline 2023-07-14T12:02:39.079887+0000) 2023-07-14T12:02:59.862 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:59 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:59.453+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.081614+0000 front 2023-07-14T12:02:23.081691+0000 (oldest deadline 2023-07-14T12:02:47.181738+0000) 2023-07-14T12:02:59.862 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:02:59 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:02:59.453+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:27.182181+0000 front 2023-07-14T12:02:27.182198+0000 (oldest deadline 2023-07-14T12:02:54.182946+0000) 2023-07-14T12:03:00.143 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:59 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:59.818+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:03:00.143 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:59 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:59.818+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:05.537048+0000 front 2023-07-14T12:02:05.536734+0000 (oldest deadline 2023-07-14T12:02:30.836845+0000) 2023-07-14T12:03:00.144 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:59 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:59.818+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:13.738547+0000 front 2023-07-14T12:02:13.738532+0000 (oldest deadline 2023-07-14T12:02:39.038483+0000) 2023-07-14T12:03:00.144 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:59 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:59.818+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:24.339615+0000 front 2023-07-14T12:02:24.339656+0000 (oldest deadline 2023-07-14T12:02:46.039911+0000) 2023-07-14T12:03:00.144 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:02:59 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:02:59.818+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.841753+0000 front 2023-07-14T12:02:31.841888+0000 (oldest deadline 2023-07-14T12:02:55.941910+0000) 2023-07-14T12:03:00.625 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:00 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:00.384+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:03:00.626 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:00 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:00.384+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:15.551211+0000 front 2023-07-14T12:02:15.551186+0000 (oldest deadline 2023-07-14T12:02:40.851263+0000) 2023-07-14T12:03:00.626 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:00 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:00.384+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:21.852804+0000 front 2023-07-14T12:02:21.852675+0000 (oldest deadline 2023-07-14T12:02:46.552594+0000) 2023-07-14T12:03:00.626 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:00 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:00.384+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.253639+0000 front 2023-07-14T12:02:31.253703+0000 (oldest deadline 2023-07-14T12:02:55.953589+0000) 2023-07-14T12:03:00.859 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:00 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:00.421+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:03:00.860 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:00 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:00.421+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.378517+0000 front 2023-07-14T12:02:07.378538+0000 (oldest deadline 2023-07-14T12:02:34.979250+0000) 2023-07-14T12:03:00.860 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:00 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:00.421+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:14.979892+0000 front 2023-07-14T12:02:14.979966+0000 (oldest deadline 2023-07-14T12:02:39.079887+0000) 2023-07-14T12:03:00.860 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:00 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:00.421+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.081614+0000 front 2023-07-14T12:02:23.081691+0000 (oldest deadline 2023-07-14T12:02:47.181738+0000) 2023-07-14T12:03:00.861 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:00 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:00.421+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:27.182181+0000 front 2023-07-14T12:02:27.182198+0000 (oldest deadline 2023-07-14T12:02:54.182946+0000) 2023-07-14T12:03:00.883 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:03:00 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:03:00.646+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:06.686677+0000 front 2023-07-14T12:02:06.686702+0000 (oldest deadline 2023-07-14T12:02:29.586617+0000) 2023-07-14T12:03:00.883 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:03:00 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:03:00.646+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:11.887528+0000 front 2023-07-14T12:02:11.887582+0000 (oldest deadline 2023-07-14T12:02:37.187740+0000) 2023-07-14T12:03:00.884 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:03:00 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:03:00.646+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.089032+0000 front 2023-07-14T12:02:23.088900+0000 (oldest deadline 2023-07-14T12:02:47.188933+0000) 2023-07-14T12:03:00.884 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:03:00 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:03:00.646+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.290100+0000 front 2023-07-14T12:02:31.290230+0000 (oldest deadline 2023-07-14T12:02:55.390035+0000) 2023-07-14T12:03:01.143 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:00 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:00.806+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:03:01.143 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:00 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:00.806+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:05.537048+0000 front 2023-07-14T12:02:05.536734+0000 (oldest deadline 2023-07-14T12:02:30.836845+0000) 2023-07-14T12:03:01.143 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:00 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:00.806+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:13.738547+0000 front 2023-07-14T12:02:13.738532+0000 (oldest deadline 2023-07-14T12:02:39.038483+0000) 2023-07-14T12:03:01.144 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:00 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:00.806+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:24.339615+0000 front 2023-07-14T12:02:24.339656+0000 (oldest deadline 2023-07-14T12:02:46.039911+0000) 2023-07-14T12:03:01.144 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:00 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:00.806+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.841753+0000 front 2023-07-14T12:02:31.841888+0000 (oldest deadline 2023-07-14T12:02:55.941910+0000) 2023-07-14T12:03:01.859 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:01 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:01.408+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:03:01.860 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:01 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:01.408+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.378517+0000 front 2023-07-14T12:02:07.378538+0000 (oldest deadline 2023-07-14T12:02:34.979250+0000) 2023-07-14T12:03:01.860 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:01 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:01.408+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:14.979892+0000 front 2023-07-14T12:02:14.979966+0000 (oldest deadline 2023-07-14T12:02:39.079887+0000) 2023-07-14T12:03:01.860 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:01 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:01.408+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.081614+0000 front 2023-07-14T12:02:23.081691+0000 (oldest deadline 2023-07-14T12:02:47.181738+0000) 2023-07-14T12:03:01.861 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:01 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:01.408+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:27.182181+0000 front 2023-07-14T12:02:27.182198+0000 (oldest deadline 2023-07-14T12:02:54.182946+0000) 2023-07-14T12:03:01.861 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:01 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:01.408+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:37.584300+0000 front 2023-07-14T12:02:37.584352+0000 (oldest deadline 2023-07-14T12:03:00.984630+0000) 2023-07-14T12:03:01.883 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:01 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:01.418+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:03:01.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:01 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:01.418+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:15.551211+0000 front 2023-07-14T12:02:15.551186+0000 (oldest deadline 2023-07-14T12:02:40.851263+0000) 2023-07-14T12:03:01.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:01 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:01.418+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:21.852804+0000 front 2023-07-14T12:02:21.852675+0000 (oldest deadline 2023-07-14T12:02:46.552594+0000) 2023-07-14T12:03:01.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:01 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:01.418+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.253639+0000 front 2023-07-14T12:02:31.253703+0000 (oldest deadline 2023-07-14T12:02:55.953589+0000) 2023-07-14T12:03:01.884 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:03:01 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:03:01.626+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:06.686677+0000 front 2023-07-14T12:02:06.686702+0000 (oldest deadline 2023-07-14T12:02:29.586617+0000) 2023-07-14T12:03:01.885 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:03:01 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:03:01.626+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:11.887528+0000 front 2023-07-14T12:02:11.887582+0000 (oldest deadline 2023-07-14T12:02:37.187740+0000) 2023-07-14T12:03:01.885 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:03:01 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:03:01.626+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.089032+0000 front 2023-07-14T12:02:23.088900+0000 (oldest deadline 2023-07-14T12:02:47.188933+0000) 2023-07-14T12:03:01.885 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:03:01 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:03:01.626+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.290100+0000 front 2023-07-14T12:02:31.290230+0000 (oldest deadline 2023-07-14T12:02:55.390035+0000) 2023-07-14T12:03:01.885 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:03:01 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:03:01.626+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:35.390357+0000 front 2023-07-14T12:02:35.390373+0000 (oldest deadline 2023-07-14T12:03:01.290428+0000) 2023-07-14T12:03:02.143 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:01 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:01.800+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:03:02.143 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:01 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:01.800+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:05.537048+0000 front 2023-07-14T12:02:05.536734+0000 (oldest deadline 2023-07-14T12:02:30.836845+0000) 2023-07-14T12:03:02.143 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:01 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:01.800+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:13.738547+0000 front 2023-07-14T12:02:13.738532+0000 (oldest deadline 2023-07-14T12:02:39.038483+0000) 2023-07-14T12:03:02.144 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:01 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:01.800+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:24.339615+0000 front 2023-07-14T12:02:24.339656+0000 (oldest deadline 2023-07-14T12:02:46.039911+0000) 2023-07-14T12:03:02.144 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:01 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:01.800+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.841753+0000 front 2023-07-14T12:02:31.841888+0000 (oldest deadline 2023-07-14T12:02:55.941910+0000) 2023-07-14T12:03:02.859 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:02 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:02.392+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:03:02.860 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:02 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:02.392+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.378517+0000 front 2023-07-14T12:02:07.378538+0000 (oldest deadline 2023-07-14T12:02:34.979250+0000) 2023-07-14T12:03:02.861 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:02 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:02.392+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:14.979892+0000 front 2023-07-14T12:02:14.979966+0000 (oldest deadline 2023-07-14T12:02:39.079887+0000) 2023-07-14T12:03:02.861 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:02 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:02.392+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.081614+0000 front 2023-07-14T12:02:23.081691+0000 (oldest deadline 2023-07-14T12:02:47.181738+0000) 2023-07-14T12:03:02.861 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:02 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:02.392+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:27.182181+0000 front 2023-07-14T12:02:27.182198+0000 (oldest deadline 2023-07-14T12:02:54.182946+0000) 2023-07-14T12:03:02.861 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:02 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:02.392+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:37.584300+0000 front 2023-07-14T12:02:37.584352+0000 (oldest deadline 2023-07-14T12:03:00.984630+0000) 2023-07-14T12:03:02.883 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:03:02 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:03:02.591+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:06.686677+0000 front 2023-07-14T12:02:06.686702+0000 (oldest deadline 2023-07-14T12:02:29.586617+0000) 2023-07-14T12:03:02.883 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:03:02 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:03:02.591+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:11.887528+0000 front 2023-07-14T12:02:11.887582+0000 (oldest deadline 2023-07-14T12:02:37.187740+0000) 2023-07-14T12:03:02.884 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:03:02 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:03:02.591+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.089032+0000 front 2023-07-14T12:02:23.088900+0000 (oldest deadline 2023-07-14T12:02:47.188933+0000) 2023-07-14T12:03:02.884 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:03:02 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:03:02.591+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.290100+0000 front 2023-07-14T12:02:31.290230+0000 (oldest deadline 2023-07-14T12:02:55.390035+0000) 2023-07-14T12:03:02.884 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:03:02 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:03:02.591+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:35.390357+0000 front 2023-07-14T12:02:35.390373+0000 (oldest deadline 2023-07-14T12:03:01.290428+0000) 2023-07-14T12:03:02.885 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:02 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:02.461+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:03:02.885 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:02 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:02.461+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:15.551211+0000 front 2023-07-14T12:02:15.551186+0000 (oldest deadline 2023-07-14T12:02:40.851263+0000) 2023-07-14T12:03:02.886 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:02 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:02.461+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:21.852804+0000 front 2023-07-14T12:02:21.852675+0000 (oldest deadline 2023-07-14T12:02:46.552594+0000) 2023-07-14T12:03:02.886 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:02 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:02.461+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.253639+0000 front 2023-07-14T12:02:31.253703+0000 (oldest deadline 2023-07-14T12:02:55.953589+0000) 2023-07-14T12:03:02.886 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:02 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:02.461+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:37.054551+0000 front 2023-07-14T12:02:37.054662+0000 (oldest deadline 2023-07-14T12:03:01.754646+0000) 2023-07-14T12:03:03.143 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:02 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:02.824+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:03:03.144 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:02 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:02.824+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:05.537048+0000 front 2023-07-14T12:02:05.536734+0000 (oldest deadline 2023-07-14T12:02:30.836845+0000) 2023-07-14T12:03:03.144 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:02 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:02.824+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:13.738547+0000 front 2023-07-14T12:02:13.738532+0000 (oldest deadline 2023-07-14T12:02:39.038483+0000) 2023-07-14T12:03:03.144 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:02 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:02.824+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:24.339615+0000 front 2023-07-14T12:02:24.339656+0000 (oldest deadline 2023-07-14T12:02:46.039911+0000) 2023-07-14T12:03:03.145 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:02 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:02.824+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.841753+0000 front 2023-07-14T12:02:31.841888+0000 (oldest deadline 2023-07-14T12:02:55.941910+0000) 2023-07-14T12:03:03.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:03 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:03.345+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:03:03.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:03 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:03.345+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.378517+0000 front 2023-07-14T12:02:07.378538+0000 (oldest deadline 2023-07-14T12:02:34.979250+0000) 2023-07-14T12:03:03.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:03 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:03.345+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:14.979892+0000 front 2023-07-14T12:02:14.979966+0000 (oldest deadline 2023-07-14T12:02:39.079887+0000) 2023-07-14T12:03:03.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:03 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:03.345+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.081614+0000 front 2023-07-14T12:02:23.081691+0000 (oldest deadline 2023-07-14T12:02:47.181738+0000) 2023-07-14T12:03:03.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:03 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:03.345+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:27.182181+0000 front 2023-07-14T12:02:27.182198+0000 (oldest deadline 2023-07-14T12:02:54.182946+0000) 2023-07-14T12:03:03.612 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:03 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:03.345+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:37.584300+0000 front 2023-07-14T12:02:37.584352+0000 (oldest deadline 2023-07-14T12:03:00.984630+0000) 2023-07-14T12:03:03.883 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:03:03 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:03:03.627+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:06.686677+0000 front 2023-07-14T12:02:06.686702+0000 (oldest deadline 2023-07-14T12:02:29.586617+0000) 2023-07-14T12:03:03.884 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:03:03 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:03:03.627+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:11.887528+0000 front 2023-07-14T12:02:11.887582+0000 (oldest deadline 2023-07-14T12:02:37.187740+0000) 2023-07-14T12:03:03.884 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:03:03 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:03:03.627+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.089032+0000 front 2023-07-14T12:02:23.088900+0000 (oldest deadline 2023-07-14T12:02:47.188933+0000) 2023-07-14T12:03:03.884 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:03:03 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:03:03.627+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.290100+0000 front 2023-07-14T12:02:31.290230+0000 (oldest deadline 2023-07-14T12:02:55.390035+0000) 2023-07-14T12:03:03.884 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:03:03 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8[146148]: 2023-07-14T12:03:03.627+0000 7ff861877700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:35.390357+0000 front 2023-07-14T12:02:35.390373+0000 (oldest deadline 2023-07-14T12:03:01.290428+0000) 2023-07-14T12:03:03.885 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:03 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:03.479+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:03:03.885 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:03 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:03.479+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:15.551211+0000 front 2023-07-14T12:02:15.551186+0000 (oldest deadline 2023-07-14T12:02:40.851263+0000) 2023-07-14T12:03:03.885 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:03 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:03.479+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:21.852804+0000 front 2023-07-14T12:02:21.852675+0000 (oldest deadline 2023-07-14T12:02:46.552594+0000) 2023-07-14T12:03:03.885 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:03 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:03.479+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.253639+0000 front 2023-07-14T12:02:31.253703+0000 (oldest deadline 2023-07-14T12:02:55.953589+0000) 2023-07-14T12:03:03.885 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:03 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:03.479+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:37.054551+0000 front 2023-07-14T12:02:37.054662+0000 (oldest deadline 2023-07-14T12:03:01.754646+0000) 2023-07-14T12:03:04.143 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:03 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:03.839+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:03:04.144 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:03 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:03.839+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:05.537048+0000 front 2023-07-14T12:02:05.536734+0000 (oldest deadline 2023-07-14T12:02:30.836845+0000) 2023-07-14T12:03:04.144 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:03 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:03.839+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:13.738547+0000 front 2023-07-14T12:02:13.738532+0000 (oldest deadline 2023-07-14T12:02:39.038483+0000) 2023-07-14T12:03:04.144 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:03 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:03.839+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:24.339615+0000 front 2023-07-14T12:02:24.339656+0000 (oldest deadline 2023-07-14T12:02:46.039911+0000) 2023-07-14T12:03:04.144 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:03 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:03.839+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.841753+0000 front 2023-07-14T12:02:31.841888+0000 (oldest deadline 2023-07-14T12:02:55.941910+0000) 2023-07-14T12:03:04.144 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:03 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:03.839+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:40.043193+0000 front 2023-07-14T12:02:40.043144+0000 (oldest deadline 2023-07-14T12:03:03.542816+0000) 2023-07-14T12:03:04.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:04 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:04.378+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:03:04.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:04 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:04.378+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.378517+0000 front 2023-07-14T12:02:07.378538+0000 (oldest deadline 2023-07-14T12:02:34.979250+0000) 2023-07-14T12:03:04.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:04 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:04.378+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:14.979892+0000 front 2023-07-14T12:02:14.979966+0000 (oldest deadline 2023-07-14T12:02:39.079887+0000) 2023-07-14T12:03:04.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:04 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:04.378+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.081614+0000 front 2023-07-14T12:02:23.081691+0000 (oldest deadline 2023-07-14T12:02:47.181738+0000) 2023-07-14T12:03:04.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:04 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:04.378+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:27.182181+0000 front 2023-07-14T12:02:27.182198+0000 (oldest deadline 2023-07-14T12:02:54.182946+0000) 2023-07-14T12:03:04.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:04 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:04.378+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:37.584300+0000 front 2023-07-14T12:02:37.584352+0000 (oldest deadline 2023-07-14T12:03:00.984630+0000) 2023-07-14T12:03:04.771 INFO:journalctl@ceph.osd.8.smithi097.stdout:Jul 14 12:03:04 smithi097 bash[158435]: ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-8 2023-07-14T12:03:04.771 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:04 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:04.469+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:03:04.772 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:04 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:04.469+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:15.551211+0000 front 2023-07-14T12:02:15.551186+0000 (oldest deadline 2023-07-14T12:02:40.851263+0000) 2023-07-14T12:03:04.772 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:04 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:04.469+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:21.852804+0000 front 2023-07-14T12:02:21.852675+0000 (oldest deadline 2023-07-14T12:02:46.552594+0000) 2023-07-14T12:03:04.772 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:04 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:04.469+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.253639+0000 front 2023-07-14T12:02:31.253703+0000 (oldest deadline 2023-07-14T12:02:55.953589+0000) 2023-07-14T12:03:04.772 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:04 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:04.469+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:37.054551+0000 front 2023-07-14T12:02:37.054662+0000 (oldest deadline 2023-07-14T12:03:01.754646+0000) 2023-07-14T12:03:05.143 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:04 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:04.869+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:03:05.144 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:04 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:04.869+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:05.537048+0000 front 2023-07-14T12:02:05.536734+0000 (oldest deadline 2023-07-14T12:02:30.836845+0000) 2023-07-14T12:03:05.144 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:04 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:04.869+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:13.738547+0000 front 2023-07-14T12:02:13.738532+0000 (oldest deadline 2023-07-14T12:02:39.038483+0000) 2023-07-14T12:03:05.144 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:04 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:04.869+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:24.339615+0000 front 2023-07-14T12:02:24.339656+0000 (oldest deadline 2023-07-14T12:02:46.039911+0000) 2023-07-14T12:03:05.145 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:04 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:04.869+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.841753+0000 front 2023-07-14T12:02:31.841888+0000 (oldest deadline 2023-07-14T12:02:55.941910+0000) 2023-07-14T12:03:05.145 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:04 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:04.869+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:40.043193+0000 front 2023-07-14T12:02:40.043144+0000 (oldest deadline 2023-07-14T12:03:03.542816+0000) 2023-07-14T12:03:05.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:05 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:05.380+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:03:05.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:05 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:05.380+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.378517+0000 front 2023-07-14T12:02:07.378538+0000 (oldest deadline 2023-07-14T12:02:34.979250+0000) 2023-07-14T12:03:05.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:05 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:05.380+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:14.979892+0000 front 2023-07-14T12:02:14.979966+0000 (oldest deadline 2023-07-14T12:02:39.079887+0000) 2023-07-14T12:03:05.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:05 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:05.380+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.081614+0000 front 2023-07-14T12:02:23.081691+0000 (oldest deadline 2023-07-14T12:02:47.181738+0000) 2023-07-14T12:03:05.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:05 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:05.380+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:27.182181+0000 front 2023-07-14T12:02:27.182198+0000 (oldest deadline 2023-07-14T12:02:54.182946+0000) 2023-07-14T12:03:05.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:05 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:05.380+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:37.584300+0000 front 2023-07-14T12:02:37.584352+0000 (oldest deadline 2023-07-14T12:03:00.984630+0000) 2023-07-14T12:03:05.633 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:05 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:05.474+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:03:05.634 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:05 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:05.474+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:15.551211+0000 front 2023-07-14T12:02:15.551186+0000 (oldest deadline 2023-07-14T12:02:40.851263+0000) 2023-07-14T12:03:05.634 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:05 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:05.474+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:21.852804+0000 front 2023-07-14T12:02:21.852675+0000 (oldest deadline 2023-07-14T12:02:46.552594+0000) 2023-07-14T12:03:05.634 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:05 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:05.474+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.253639+0000 front 2023-07-14T12:02:31.253703+0000 (oldest deadline 2023-07-14T12:02:55.953589+0000) 2023-07-14T12:03:05.634 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:05 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:05.474+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:37.054551+0000 front 2023-07-14T12:02:37.054662+0000 (oldest deadline 2023-07-14T12:03:01.754646+0000) 2023-07-14T12:03:06.034 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@osd.8.service' 2023-07-14T12:03:06.114 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T12:03:06.115 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-07-14T12:03:06.115 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-07-14T12:03:06.116 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@osd.9 2023-07-14T12:03:06.143 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:05 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:05.858+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:03:06.143 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:05 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:05.858+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:05.537048+0000 front 2023-07-14T12:02:05.536734+0000 (oldest deadline 2023-07-14T12:02:30.836845+0000) 2023-07-14T12:03:06.143 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:05 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:05.858+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:13.738547+0000 front 2023-07-14T12:02:13.738532+0000 (oldest deadline 2023-07-14T12:02:39.038483+0000) 2023-07-14T12:03:06.143 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:05 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:05.858+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:24.339615+0000 front 2023-07-14T12:02:24.339656+0000 (oldest deadline 2023-07-14T12:02:46.039911+0000) 2023-07-14T12:03:06.144 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:05 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:05.858+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.841753+0000 front 2023-07-14T12:02:31.841888+0000 (oldest deadline 2023-07-14T12:02:55.941910+0000) 2023-07-14T12:03:06.144 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:05 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:05.858+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:40.043193+0000 front 2023-07-14T12:02:40.043144+0000 (oldest deadline 2023-07-14T12:03:03.542816+0000) 2023-07-14T12:03:06.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:06 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:06.352+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:03:06.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:06 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:06.352+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.378517+0000 front 2023-07-14T12:02:07.378538+0000 (oldest deadline 2023-07-14T12:02:34.979250+0000) 2023-07-14T12:03:06.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:06 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:06.352+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:14.979892+0000 front 2023-07-14T12:02:14.979966+0000 (oldest deadline 2023-07-14T12:02:39.079887+0000) 2023-07-14T12:03:06.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:06 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:06.352+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.081614+0000 front 2023-07-14T12:02:23.081691+0000 (oldest deadline 2023-07-14T12:02:47.181738+0000) 2023-07-14T12:03:06.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:06 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:06.352+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:27.182181+0000 front 2023-07-14T12:02:27.182198+0000 (oldest deadline 2023-07-14T12:02:54.182946+0000) 2023-07-14T12:03:06.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:06 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:06.352+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:37.584300+0000 front 2023-07-14T12:02:37.584352+0000 (oldest deadline 2023-07-14T12:03:00.984630+0000) 2023-07-14T12:03:06.643 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:06 smithi017 systemd[1]: Stopping Ceph osd.9 for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T12:03:06.883 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:06 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:06.471+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:03:06.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:06 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:06.471+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:15.551211+0000 front 2023-07-14T12:02:15.551186+0000 (oldest deadline 2023-07-14T12:02:40.851263+0000) 2023-07-14T12:03:06.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:06 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:06.471+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:21.852804+0000 front 2023-07-14T12:02:21.852675+0000 (oldest deadline 2023-07-14T12:02:46.552594+0000) 2023-07-14T12:03:06.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:06 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:06.471+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.253639+0000 front 2023-07-14T12:02:31.253703+0000 (oldest deadline 2023-07-14T12:02:55.953589+0000) 2023-07-14T12:03:06.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:06 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:06.471+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:37.054551+0000 front 2023-07-14T12:02:37.054662+0000 (oldest deadline 2023-07-14T12:03:01.754646+0000) 2023-07-14T12:03:07.143 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:06 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:06.664+0000 7fbdda5ff700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T12:03:07.143 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:06 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:06.664+0000 7fbdda5ff700 -1 osd.9 96 *** Got signal Terminated *** 2023-07-14T12:03:07.143 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:06 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:06.664+0000 7fbdda5ff700 -1 osd.9 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T12:03:07.144 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:06 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:06.814+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:03:07.144 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:06 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:06.814+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:05.537048+0000 front 2023-07-14T12:02:05.536734+0000 (oldest deadline 2023-07-14T12:02:30.836845+0000) 2023-07-14T12:03:07.144 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:06 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:06.814+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:13.738547+0000 front 2023-07-14T12:02:13.738532+0000 (oldest deadline 2023-07-14T12:02:39.038483+0000) 2023-07-14T12:03:07.145 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:06 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:06.814+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:24.339615+0000 front 2023-07-14T12:02:24.339656+0000 (oldest deadline 2023-07-14T12:02:46.039911+0000) 2023-07-14T12:03:07.145 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:06 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:06.814+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.841753+0000 front 2023-07-14T12:02:31.841888+0000 (oldest deadline 2023-07-14T12:02:55.941910+0000) 2023-07-14T12:03:07.145 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:06 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:06.814+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:40.043193+0000 front 2023-07-14T12:02:40.043144+0000 (oldest deadline 2023-07-14T12:03:03.542816+0000) 2023-07-14T12:03:07.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:07 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:07.325+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:03:07.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:07 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:07.325+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.378517+0000 front 2023-07-14T12:02:07.378538+0000 (oldest deadline 2023-07-14T12:02:34.979250+0000) 2023-07-14T12:03:07.612 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:07 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:07.325+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:14.979892+0000 front 2023-07-14T12:02:14.979966+0000 (oldest deadline 2023-07-14T12:02:39.079887+0000) 2023-07-14T12:03:07.612 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:07 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:07.325+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.081614+0000 front 2023-07-14T12:02:23.081691+0000 (oldest deadline 2023-07-14T12:02:47.181738+0000) 2023-07-14T12:03:07.612 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:07 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:07.325+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:27.182181+0000 front 2023-07-14T12:02:27.182198+0000 (oldest deadline 2023-07-14T12:02:54.182946+0000) 2023-07-14T12:03:07.612 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:07 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:07.325+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:37.584300+0000 front 2023-07-14T12:02:37.584352+0000 (oldest deadline 2023-07-14T12:03:00.984630+0000) 2023-07-14T12:03:07.883 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:07 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:07.496+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:03:07.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:07 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:07.496+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:15.551211+0000 front 2023-07-14T12:02:15.551186+0000 (oldest deadline 2023-07-14T12:02:40.851263+0000) 2023-07-14T12:03:07.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:07 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:07.496+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:21.852804+0000 front 2023-07-14T12:02:21.852675+0000 (oldest deadline 2023-07-14T12:02:46.552594+0000) 2023-07-14T12:03:07.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:07 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:07.496+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.253639+0000 front 2023-07-14T12:02:31.253703+0000 (oldest deadline 2023-07-14T12:02:55.953589+0000) 2023-07-14T12:03:07.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:07 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:07.496+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:37.054551+0000 front 2023-07-14T12:02:37.054662+0000 (oldest deadline 2023-07-14T12:03:01.754646+0000) 2023-07-14T12:03:08.143 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:07 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:07.860+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:03:08.143 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:07 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:07.860+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:05.537048+0000 front 2023-07-14T12:02:05.536734+0000 (oldest deadline 2023-07-14T12:02:30.836845+0000) 2023-07-14T12:03:08.144 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:07 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:07.860+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:13.738547+0000 front 2023-07-14T12:02:13.738532+0000 (oldest deadline 2023-07-14T12:02:39.038483+0000) 2023-07-14T12:03:08.144 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:07 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:07.860+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:24.339615+0000 front 2023-07-14T12:02:24.339656+0000 (oldest deadline 2023-07-14T12:02:46.039911+0000) 2023-07-14T12:03:08.144 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:07 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:07.860+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.841753+0000 front 2023-07-14T12:02:31.841888+0000 (oldest deadline 2023-07-14T12:02:55.941910+0000) 2023-07-14T12:03:08.144 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:07 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:07.860+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:40.043193+0000 front 2023-07-14T12:02:40.043144+0000 (oldest deadline 2023-07-14T12:03:03.542816+0000) 2023-07-14T12:03:08.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:08 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:08.325+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:03:08.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:08 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:08.325+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.378517+0000 front 2023-07-14T12:02:07.378538+0000 (oldest deadline 2023-07-14T12:02:34.979250+0000) 2023-07-14T12:03:08.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:08 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:08.325+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:14.979892+0000 front 2023-07-14T12:02:14.979966+0000 (oldest deadline 2023-07-14T12:02:39.079887+0000) 2023-07-14T12:03:08.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:08 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:08.325+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.081614+0000 front 2023-07-14T12:02:23.081691+0000 (oldest deadline 2023-07-14T12:02:47.181738+0000) 2023-07-14T12:03:08.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:08 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:08.325+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:27.182181+0000 front 2023-07-14T12:02:27.182198+0000 (oldest deadline 2023-07-14T12:02:54.182946+0000) 2023-07-14T12:03:08.612 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:08 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:08.325+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:37.584300+0000 front 2023-07-14T12:02:37.584352+0000 (oldest deadline 2023-07-14T12:03:00.984630+0000) 2023-07-14T12:03:08.883 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:08 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:08.546+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:03:08.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:08 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:08.546+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:15.551211+0000 front 2023-07-14T12:02:15.551186+0000 (oldest deadline 2023-07-14T12:02:40.851263+0000) 2023-07-14T12:03:08.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:08 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:08.546+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:21.852804+0000 front 2023-07-14T12:02:21.852675+0000 (oldest deadline 2023-07-14T12:02:46.552594+0000) 2023-07-14T12:03:08.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:08 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:08.546+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.253639+0000 front 2023-07-14T12:02:31.253703+0000 (oldest deadline 2023-07-14T12:02:55.953589+0000) 2023-07-14T12:03:08.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:08 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:08.546+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:37.054551+0000 front 2023-07-14T12:02:37.054662+0000 (oldest deadline 2023-07-14T12:03:01.754646+0000) 2023-07-14T12:03:09.143 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:08 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:08.818+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:03:09.143 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:08 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:08.818+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:05.537048+0000 front 2023-07-14T12:02:05.536734+0000 (oldest deadline 2023-07-14T12:02:30.836845+0000) 2023-07-14T12:03:09.144 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:08 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:08.818+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:13.738547+0000 front 2023-07-14T12:02:13.738532+0000 (oldest deadline 2023-07-14T12:02:39.038483+0000) 2023-07-14T12:03:09.144 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:08 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:08.818+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:24.339615+0000 front 2023-07-14T12:02:24.339656+0000 (oldest deadline 2023-07-14T12:02:46.039911+0000) 2023-07-14T12:03:09.144 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:08 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:08.818+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.841753+0000 front 2023-07-14T12:02:31.841888+0000 (oldest deadline 2023-07-14T12:02:55.941910+0000) 2023-07-14T12:03:09.145 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:08 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:08.818+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:40.043193+0000 front 2023-07-14T12:02:40.043144+0000 (oldest deadline 2023-07-14T12:03:03.542816+0000) 2023-07-14T12:03:09.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:09 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:09.301+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:03:09.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:09 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:09.301+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.378517+0000 front 2023-07-14T12:02:07.378538+0000 (oldest deadline 2023-07-14T12:02:34.979250+0000) 2023-07-14T12:03:09.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:09 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:09.301+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:14.979892+0000 front 2023-07-14T12:02:14.979966+0000 (oldest deadline 2023-07-14T12:02:39.079887+0000) 2023-07-14T12:03:09.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:09 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:09.301+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.081614+0000 front 2023-07-14T12:02:23.081691+0000 (oldest deadline 2023-07-14T12:02:47.181738+0000) 2023-07-14T12:03:09.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:09 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:09.301+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:27.182181+0000 front 2023-07-14T12:02:27.182198+0000 (oldest deadline 2023-07-14T12:02:54.182946+0000) 2023-07-14T12:03:09.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:09 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:09.301+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:37.584300+0000 front 2023-07-14T12:02:37.584352+0000 (oldest deadline 2023-07-14T12:03:00.984630+0000) 2023-07-14T12:03:09.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:09 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:09.301+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2023-07-14T12:02:43.786376+0000 front 2023-07-14T12:02:43.786425+0000 (oldest deadline 2023-07-14T12:03:08.486282+0000) 2023-07-14T12:03:09.883 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:09 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:09.565+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:03:09.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:09 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:09.565+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:15.551211+0000 front 2023-07-14T12:02:15.551186+0000 (oldest deadline 2023-07-14T12:02:40.851263+0000) 2023-07-14T12:03:09.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:09 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:09.565+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:21.852804+0000 front 2023-07-14T12:02:21.852675+0000 (oldest deadline 2023-07-14T12:02:46.552594+0000) 2023-07-14T12:03:09.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:09 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:09.565+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.253639+0000 front 2023-07-14T12:02:31.253703+0000 (oldest deadline 2023-07-14T12:02:55.953589+0000) 2023-07-14T12:03:09.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:09 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:09.565+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:37.054551+0000 front 2023-07-14T12:02:37.054662+0000 (oldest deadline 2023-07-14T12:03:01.754646+0000) 2023-07-14T12:03:10.143 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:09 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:09.794+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:03:10.143 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:09 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:09.794+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:05.537048+0000 front 2023-07-14T12:02:05.536734+0000 (oldest deadline 2023-07-14T12:02:30.836845+0000) 2023-07-14T12:03:10.144 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:09 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:09.794+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:13.738547+0000 front 2023-07-14T12:02:13.738532+0000 (oldest deadline 2023-07-14T12:02:39.038483+0000) 2023-07-14T12:03:10.144 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:09 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:09.794+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:24.339615+0000 front 2023-07-14T12:02:24.339656+0000 (oldest deadline 2023-07-14T12:02:46.039911+0000) 2023-07-14T12:03:10.144 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:09 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:09.794+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.841753+0000 front 2023-07-14T12:02:31.841888+0000 (oldest deadline 2023-07-14T12:02:55.941910+0000) 2023-07-14T12:03:10.144 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:09 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:09.794+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:40.043193+0000 front 2023-07-14T12:02:40.043144+0000 (oldest deadline 2023-07-14T12:03:03.542816+0000) 2023-07-14T12:03:10.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:10 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:10.319+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:03:10.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:10 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:10.319+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.378517+0000 front 2023-07-14T12:02:07.378538+0000 (oldest deadline 2023-07-14T12:02:34.979250+0000) 2023-07-14T12:03:10.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:10 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:10.319+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:14.979892+0000 front 2023-07-14T12:02:14.979966+0000 (oldest deadline 2023-07-14T12:02:39.079887+0000) 2023-07-14T12:03:10.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:10 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:10.319+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.081614+0000 front 2023-07-14T12:02:23.081691+0000 (oldest deadline 2023-07-14T12:02:47.181738+0000) 2023-07-14T12:03:10.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:10 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:10.319+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:27.182181+0000 front 2023-07-14T12:02:27.182198+0000 (oldest deadline 2023-07-14T12:02:54.182946+0000) 2023-07-14T12:03:10.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:10 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:10.319+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:37.584300+0000 front 2023-07-14T12:02:37.584352+0000 (oldest deadline 2023-07-14T12:03:00.984630+0000) 2023-07-14T12:03:10.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:10 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:10.319+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2023-07-14T12:02:43.786376+0000 front 2023-07-14T12:02:43.786425+0000 (oldest deadline 2023-07-14T12:03:08.486282+0000) 2023-07-14T12:03:10.883 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:10 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:10.605+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:03:10.883 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:10 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:10.605+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:15.551211+0000 front 2023-07-14T12:02:15.551186+0000 (oldest deadline 2023-07-14T12:02:40.851263+0000) 2023-07-14T12:03:10.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:10 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:10.605+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:21.852804+0000 front 2023-07-14T12:02:21.852675+0000 (oldest deadline 2023-07-14T12:02:46.552594+0000) 2023-07-14T12:03:10.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:10 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:10.605+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.253639+0000 front 2023-07-14T12:02:31.253703+0000 (oldest deadline 2023-07-14T12:02:55.953589+0000) 2023-07-14T12:03:10.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:10 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:10.605+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:37.054551+0000 front 2023-07-14T12:02:37.054662+0000 (oldest deadline 2023-07-14T12:03:01.754646+0000) 2023-07-14T12:03:10.885 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:10 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:10.605+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2023-07-14T12:02:45.255728+0000 front 2023-07-14T12:02:45.255708+0000 (oldest deadline 2023-07-14T12:03:10.555481+0000) 2023-07-14T12:03:11.143 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:10 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:10.836+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.135134+0000 front 2023-07-14T12:01:59.135187+0000 (oldest deadline 2023-07-14T12:02:22.635230+0000) 2023-07-14T12:03:11.144 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:10 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:10.836+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:05.537048+0000 front 2023-07-14T12:02:05.536734+0000 (oldest deadline 2023-07-14T12:02:30.836845+0000) 2023-07-14T12:03:11.144 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:10 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:10.836+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:13.738547+0000 front 2023-07-14T12:02:13.738532+0000 (oldest deadline 2023-07-14T12:02:39.038483+0000) 2023-07-14T12:03:11.144 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:10 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:10.836+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:24.339615+0000 front 2023-07-14T12:02:24.339656+0000 (oldest deadline 2023-07-14T12:02:46.039911+0000) 2023-07-14T12:03:11.145 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:10 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:10.836+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.841753+0000 front 2023-07-14T12:02:31.841888+0000 (oldest deadline 2023-07-14T12:02:55.941910+0000) 2023-07-14T12:03:11.145 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:10 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9[155813]: 2023-07-14T12:03:10.836+0000 7fbdd6bf5700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:40.043193+0000 front 2023-07-14T12:02:40.043144+0000 (oldest deadline 2023-07-14T12:03:03.542816+0000) 2023-07-14T12:03:11.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:11 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:11.298+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:03:11.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:11 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:11.298+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.378517+0000 front 2023-07-14T12:02:07.378538+0000 (oldest deadline 2023-07-14T12:02:34.979250+0000) 2023-07-14T12:03:11.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:11 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:11.298+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:14.979892+0000 front 2023-07-14T12:02:14.979966+0000 (oldest deadline 2023-07-14T12:02:39.079887+0000) 2023-07-14T12:03:11.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:11 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:11.298+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.081614+0000 front 2023-07-14T12:02:23.081691+0000 (oldest deadline 2023-07-14T12:02:47.181738+0000) 2023-07-14T12:03:11.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:11 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:11.298+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:27.182181+0000 front 2023-07-14T12:02:27.182198+0000 (oldest deadline 2023-07-14T12:02:54.182946+0000) 2023-07-14T12:03:11.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:11 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:11.298+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:37.584300+0000 front 2023-07-14T12:02:37.584352+0000 (oldest deadline 2023-07-14T12:03:00.984630+0000) 2023-07-14T12:03:11.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:11 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:11.298+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2023-07-14T12:02:43.786376+0000 front 2023-07-14T12:02:43.786425+0000 (oldest deadline 2023-07-14T12:03:08.486282+0000) 2023-07-14T12:03:11.883 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:11 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:11.565+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:03:11.883 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:11 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:11.565+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:15.551211+0000 front 2023-07-14T12:02:15.551186+0000 (oldest deadline 2023-07-14T12:02:40.851263+0000) 2023-07-14T12:03:11.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:11 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:11.565+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:21.852804+0000 front 2023-07-14T12:02:21.852675+0000 (oldest deadline 2023-07-14T12:02:46.552594+0000) 2023-07-14T12:03:11.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:11 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:11.565+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.253639+0000 front 2023-07-14T12:02:31.253703+0000 (oldest deadline 2023-07-14T12:02:55.953589+0000) 2023-07-14T12:03:11.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:11 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:11.565+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:37.054551+0000 front 2023-07-14T12:02:37.054662+0000 (oldest deadline 2023-07-14T12:03:01.754646+0000) 2023-07-14T12:03:11.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:11 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:11.565+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2023-07-14T12:02:45.255728+0000 front 2023-07-14T12:02:45.255708+0000 (oldest deadline 2023-07-14T12:03:10.555481+0000) 2023-07-14T12:03:12.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:12 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:12.253+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:03:12.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:12 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:12.253+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.378517+0000 front 2023-07-14T12:02:07.378538+0000 (oldest deadline 2023-07-14T12:02:34.979250+0000) 2023-07-14T12:03:12.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:12 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:12.253+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:14.979892+0000 front 2023-07-14T12:02:14.979966+0000 (oldest deadline 2023-07-14T12:02:39.079887+0000) 2023-07-14T12:03:12.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:12 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:12.253+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.081614+0000 front 2023-07-14T12:02:23.081691+0000 (oldest deadline 2023-07-14T12:02:47.181738+0000) 2023-07-14T12:03:12.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:12 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:12.253+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:27.182181+0000 front 2023-07-14T12:02:27.182198+0000 (oldest deadline 2023-07-14T12:02:54.182946+0000) 2023-07-14T12:03:12.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:12 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:12.253+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:37.584300+0000 front 2023-07-14T12:02:37.584352+0000 (oldest deadline 2023-07-14T12:03:00.984630+0000) 2023-07-14T12:03:12.612 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:12 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:12.253+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2023-07-14T12:02:43.786376+0000 front 2023-07-14T12:02:43.786425+0000 (oldest deadline 2023-07-14T12:03:08.486282+0000) 2023-07-14T12:03:12.643 INFO:journalctl@ceph.osd.9.smithi017.stdout:Jul 14 12:03:12 smithi017 bash[176798]: ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-9 2023-07-14T12:03:12.883 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:12 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:12.570+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:03:12.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:12 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:12.570+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:15.551211+0000 front 2023-07-14T12:02:15.551186+0000 (oldest deadline 2023-07-14T12:02:40.851263+0000) 2023-07-14T12:03:12.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:12 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:12.570+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:21.852804+0000 front 2023-07-14T12:02:21.852675+0000 (oldest deadline 2023-07-14T12:02:46.552594+0000) 2023-07-14T12:03:12.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:12 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:12.570+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.253639+0000 front 2023-07-14T12:02:31.253703+0000 (oldest deadline 2023-07-14T12:02:55.953589+0000) 2023-07-14T12:03:12.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:12 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:12.570+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:37.054551+0000 front 2023-07-14T12:02:37.054662+0000 (oldest deadline 2023-07-14T12:03:01.754646+0000) 2023-07-14T12:03:12.885 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:12 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:12.570+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2023-07-14T12:02:45.255728+0000 front 2023-07-14T12:02:45.255708+0000 (oldest deadline 2023-07-14T12:03:10.555481+0000) 2023-07-14T12:03:13.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:13 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:13.204+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:03:13.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:13 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:13.204+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.378517+0000 front 2023-07-14T12:02:07.378538+0000 (oldest deadline 2023-07-14T12:02:34.979250+0000) 2023-07-14T12:03:13.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:13 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:13.204+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:14.979892+0000 front 2023-07-14T12:02:14.979966+0000 (oldest deadline 2023-07-14T12:02:39.079887+0000) 2023-07-14T12:03:13.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:13 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:13.204+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.081614+0000 front 2023-07-14T12:02:23.081691+0000 (oldest deadline 2023-07-14T12:02:47.181738+0000) 2023-07-14T12:03:13.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:13 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:13.204+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:27.182181+0000 front 2023-07-14T12:02:27.182198+0000 (oldest deadline 2023-07-14T12:02:54.182946+0000) 2023-07-14T12:03:13.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:13 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:13.204+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:37.584300+0000 front 2023-07-14T12:02:37.584352+0000 (oldest deadline 2023-07-14T12:03:00.984630+0000) 2023-07-14T12:03:13.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:13 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:13.204+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2023-07-14T12:02:43.786376+0000 front 2023-07-14T12:02:43.786425+0000 (oldest deadline 2023-07-14T12:03:08.486282+0000) 2023-07-14T12:03:13.883 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:13 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:13.545+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:03:13.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:13 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:13.545+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:15.551211+0000 front 2023-07-14T12:02:15.551186+0000 (oldest deadline 2023-07-14T12:02:40.851263+0000) 2023-07-14T12:03:13.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:13 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:13.545+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:21.852804+0000 front 2023-07-14T12:02:21.852675+0000 (oldest deadline 2023-07-14T12:02:46.552594+0000) 2023-07-14T12:03:13.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:13 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:13.545+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.253639+0000 front 2023-07-14T12:02:31.253703+0000 (oldest deadline 2023-07-14T12:02:55.953589+0000) 2023-07-14T12:03:13.885 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:13 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:13.545+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:37.054551+0000 front 2023-07-14T12:02:37.054662+0000 (oldest deadline 2023-07-14T12:03:01.754646+0000) 2023-07-14T12:03:13.885 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:13 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:13.545+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2023-07-14T12:02:45.255728+0000 front 2023-07-14T12:02:45.255708+0000 (oldest deadline 2023-07-14T12:03:10.555481+0000) 2023-07-14T12:03:14.124 DEBUG:teuthology.orchestra.run.smithi017:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@osd.9.service' 2023-07-14T12:03:14.254 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T12:03:14.255 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-07-14T12:03:14.256 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-07-14T12:03:14.256 DEBUG:teuthology.orchestra.run.smithi093:> sudo systemctl stop ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@osd.10 2023-07-14T12:03:14.609 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:14 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:14.251+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:03:14.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:14 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:14.251+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.378517+0000 front 2023-07-14T12:02:07.378538+0000 (oldest deadline 2023-07-14T12:02:34.979250+0000) 2023-07-14T12:03:14.612 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:14 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:14.251+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:14.979892+0000 front 2023-07-14T12:02:14.979966+0000 (oldest deadline 2023-07-14T12:02:39.079887+0000) 2023-07-14T12:03:14.612 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:14 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:14.251+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.081614+0000 front 2023-07-14T12:02:23.081691+0000 (oldest deadline 2023-07-14T12:02:47.181738+0000) 2023-07-14T12:03:14.612 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:14 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:14.251+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:27.182181+0000 front 2023-07-14T12:02:27.182198+0000 (oldest deadline 2023-07-14T12:02:54.182946+0000) 2023-07-14T12:03:14.612 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:14 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:14.251+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:37.584300+0000 front 2023-07-14T12:02:37.584352+0000 (oldest deadline 2023-07-14T12:03:00.984630+0000) 2023-07-14T12:03:14.612 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:14 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:14.251+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2023-07-14T12:02:43.786376+0000 front 2023-07-14T12:02:43.786425+0000 (oldest deadline 2023-07-14T12:03:08.486282+0000) 2023-07-14T12:03:14.613 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:14 smithi093 systemd[1]: Stopping Ceph osd.10 for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T12:03:14.613 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:14 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:14.631+0000 7f8792bff700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T12:03:14.613 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:14 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:14.631+0000 7f8792bff700 -1 osd.10 96 *** Got signal Terminated *** 2023-07-14T12:03:14.613 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:14 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:14.631+0000 7f8792bff700 -1 osd.10 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T12:03:14.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:14 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:14.570+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:03:14.885 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:14 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:14.570+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:15.551211+0000 front 2023-07-14T12:02:15.551186+0000 (oldest deadline 2023-07-14T12:02:40.851263+0000) 2023-07-14T12:03:14.885 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:14 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:14.570+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:21.852804+0000 front 2023-07-14T12:02:21.852675+0000 (oldest deadline 2023-07-14T12:02:46.552594+0000) 2023-07-14T12:03:14.886 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:14 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:14.570+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.253639+0000 front 2023-07-14T12:02:31.253703+0000 (oldest deadline 2023-07-14T12:02:55.953589+0000) 2023-07-14T12:03:14.886 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:14 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:14.570+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:37.054551+0000 front 2023-07-14T12:02:37.054662+0000 (oldest deadline 2023-07-14T12:03:01.754646+0000) 2023-07-14T12:03:14.886 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:14 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:14.570+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2023-07-14T12:02:45.255728+0000 front 2023-07-14T12:02:45.255708+0000 (oldest deadline 2023-07-14T12:03:10.555481+0000) 2023-07-14T12:03:15.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:15 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:15.244+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:03:15.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:15 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:15.244+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.378517+0000 front 2023-07-14T12:02:07.378538+0000 (oldest deadline 2023-07-14T12:02:34.979250+0000) 2023-07-14T12:03:15.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:15 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:15.244+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:14.979892+0000 front 2023-07-14T12:02:14.979966+0000 (oldest deadline 2023-07-14T12:02:39.079887+0000) 2023-07-14T12:03:15.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:15 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:15.244+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.081614+0000 front 2023-07-14T12:02:23.081691+0000 (oldest deadline 2023-07-14T12:02:47.181738+0000) 2023-07-14T12:03:15.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:15 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:15.244+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:27.182181+0000 front 2023-07-14T12:02:27.182198+0000 (oldest deadline 2023-07-14T12:02:54.182946+0000) 2023-07-14T12:03:15.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:15 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:15.244+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:37.584300+0000 front 2023-07-14T12:02:37.584352+0000 (oldest deadline 2023-07-14T12:03:00.984630+0000) 2023-07-14T12:03:15.612 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:15 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:15.244+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2023-07-14T12:02:43.786376+0000 front 2023-07-14T12:02:43.786425+0000 (oldest deadline 2023-07-14T12:03:08.486282+0000) 2023-07-14T12:03:15.883 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:15 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:15.562+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:03:15.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:15 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:15.562+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:15.551211+0000 front 2023-07-14T12:02:15.551186+0000 (oldest deadline 2023-07-14T12:02:40.851263+0000) 2023-07-14T12:03:15.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:15 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:15.562+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:21.852804+0000 front 2023-07-14T12:02:21.852675+0000 (oldest deadline 2023-07-14T12:02:46.552594+0000) 2023-07-14T12:03:15.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:15 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:15.562+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.253639+0000 front 2023-07-14T12:02:31.253703+0000 (oldest deadline 2023-07-14T12:02:55.953589+0000) 2023-07-14T12:03:15.885 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:15 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:15.562+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:37.054551+0000 front 2023-07-14T12:02:37.054662+0000 (oldest deadline 2023-07-14T12:03:01.754646+0000) 2023-07-14T12:03:15.885 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:15 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:15.562+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2023-07-14T12:02:45.255728+0000 front 2023-07-14T12:02:45.255708+0000 (oldest deadline 2023-07-14T12:03:10.555481+0000) 2023-07-14T12:03:16.609 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:16 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:16.208+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:03:16.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:16 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:16.208+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.378517+0000 front 2023-07-14T12:02:07.378538+0000 (oldest deadline 2023-07-14T12:02:34.979250+0000) 2023-07-14T12:03:16.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:16 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:16.208+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:14.979892+0000 front 2023-07-14T12:02:14.979966+0000 (oldest deadline 2023-07-14T12:02:39.079887+0000) 2023-07-14T12:03:16.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:16 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:16.208+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.081614+0000 front 2023-07-14T12:02:23.081691+0000 (oldest deadline 2023-07-14T12:02:47.181738+0000) 2023-07-14T12:03:16.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:16 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:16.208+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:27.182181+0000 front 2023-07-14T12:02:27.182198+0000 (oldest deadline 2023-07-14T12:02:54.182946+0000) 2023-07-14T12:03:16.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:16 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:16.208+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:37.584300+0000 front 2023-07-14T12:02:37.584352+0000 (oldest deadline 2023-07-14T12:03:00.984630+0000) 2023-07-14T12:03:16.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:16 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:16.208+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2023-07-14T12:02:43.786376+0000 front 2023-07-14T12:02:43.786425+0000 (oldest deadline 2023-07-14T12:03:08.486282+0000) 2023-07-14T12:03:16.883 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:16 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:16.569+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:03:16.883 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:16 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:16.569+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:15.551211+0000 front 2023-07-14T12:02:15.551186+0000 (oldest deadline 2023-07-14T12:02:40.851263+0000) 2023-07-14T12:03:16.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:16 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:16.569+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:21.852804+0000 front 2023-07-14T12:02:21.852675+0000 (oldest deadline 2023-07-14T12:02:46.552594+0000) 2023-07-14T12:03:16.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:16 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:16.569+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.253639+0000 front 2023-07-14T12:02:31.253703+0000 (oldest deadline 2023-07-14T12:02:55.953589+0000) 2023-07-14T12:03:16.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:16 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:16.569+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:37.054551+0000 front 2023-07-14T12:02:37.054662+0000 (oldest deadline 2023-07-14T12:03:01.754646+0000) 2023-07-14T12:03:16.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:16 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:16.569+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2023-07-14T12:02:45.255728+0000 front 2023-07-14T12:02:45.255708+0000 (oldest deadline 2023-07-14T12:03:10.555481+0000) 2023-07-14T12:03:17.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:17 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:17.191+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:03:17.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:17 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:17.191+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.378517+0000 front 2023-07-14T12:02:07.378538+0000 (oldest deadline 2023-07-14T12:02:34.979250+0000) 2023-07-14T12:03:17.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:17 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:17.191+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:14.979892+0000 front 2023-07-14T12:02:14.979966+0000 (oldest deadline 2023-07-14T12:02:39.079887+0000) 2023-07-14T12:03:17.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:17 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:17.191+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.081614+0000 front 2023-07-14T12:02:23.081691+0000 (oldest deadline 2023-07-14T12:02:47.181738+0000) 2023-07-14T12:03:17.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:17 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:17.191+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:27.182181+0000 front 2023-07-14T12:02:27.182198+0000 (oldest deadline 2023-07-14T12:02:54.182946+0000) 2023-07-14T12:03:17.612 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:17 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:17.191+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:37.584300+0000 front 2023-07-14T12:02:37.584352+0000 (oldest deadline 2023-07-14T12:03:00.984630+0000) 2023-07-14T12:03:17.612 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:17 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:17.191+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2023-07-14T12:02:43.786376+0000 front 2023-07-14T12:02:43.786425+0000 (oldest deadline 2023-07-14T12:03:08.486282+0000) 2023-07-14T12:03:17.883 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:17 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:17.573+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:03:17.883 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:17 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:17.573+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:15.551211+0000 front 2023-07-14T12:02:15.551186+0000 (oldest deadline 2023-07-14T12:02:40.851263+0000) 2023-07-14T12:03:17.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:17 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:17.573+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:21.852804+0000 front 2023-07-14T12:02:21.852675+0000 (oldest deadline 2023-07-14T12:02:46.552594+0000) 2023-07-14T12:03:17.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:17 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:17.573+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.253639+0000 front 2023-07-14T12:02:31.253703+0000 (oldest deadline 2023-07-14T12:02:55.953589+0000) 2023-07-14T12:03:17.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:17 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:17.573+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:37.054551+0000 front 2023-07-14T12:02:37.054662+0000 (oldest deadline 2023-07-14T12:03:01.754646+0000) 2023-07-14T12:03:17.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:17 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:17.573+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2023-07-14T12:02:45.255728+0000 front 2023-07-14T12:02:45.255708+0000 (oldest deadline 2023-07-14T12:03:10.555481+0000) 2023-07-14T12:03:18.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:18 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:18.154+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:03:18.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:18 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:18.154+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.378517+0000 front 2023-07-14T12:02:07.378538+0000 (oldest deadline 2023-07-14T12:02:34.979250+0000) 2023-07-14T12:03:18.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:18 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:18.154+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:14.979892+0000 front 2023-07-14T12:02:14.979966+0000 (oldest deadline 2023-07-14T12:02:39.079887+0000) 2023-07-14T12:03:18.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:18 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:18.154+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.081614+0000 front 2023-07-14T12:02:23.081691+0000 (oldest deadline 2023-07-14T12:02:47.181738+0000) 2023-07-14T12:03:18.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:18 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:18.154+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:27.182181+0000 front 2023-07-14T12:02:27.182198+0000 (oldest deadline 2023-07-14T12:02:54.182946+0000) 2023-07-14T12:03:18.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:18 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:18.154+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:37.584300+0000 front 2023-07-14T12:02:37.584352+0000 (oldest deadline 2023-07-14T12:03:00.984630+0000) 2023-07-14T12:03:18.611 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:18 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:18.154+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2023-07-14T12:02:43.786376+0000 front 2023-07-14T12:02:43.786425+0000 (oldest deadline 2023-07-14T12:03:08.486282+0000) 2023-07-14T12:03:18.883 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:18 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:18.570+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:03:18.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:18 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:18.570+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:15.551211+0000 front 2023-07-14T12:02:15.551186+0000 (oldest deadline 2023-07-14T12:02:40.851263+0000) 2023-07-14T12:03:18.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:18 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:18.570+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:21.852804+0000 front 2023-07-14T12:02:21.852675+0000 (oldest deadline 2023-07-14T12:02:46.552594+0000) 2023-07-14T12:03:18.885 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:18 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:18.570+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.253639+0000 front 2023-07-14T12:02:31.253703+0000 (oldest deadline 2023-07-14T12:02:55.953589+0000) 2023-07-14T12:03:18.885 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:18 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:18.570+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:37.054551+0000 front 2023-07-14T12:02:37.054662+0000 (oldest deadline 2023-07-14T12:03:01.754646+0000) 2023-07-14T12:03:18.886 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:18 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:18.570+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2023-07-14T12:02:45.255728+0000 front 2023-07-14T12:02:45.255708+0000 (oldest deadline 2023-07-14T12:03:10.555481+0000) 2023-07-14T12:03:19.359 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:19 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:19.111+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:01:59.777868+0000 front 2023-07-14T12:01:59.777727+0000 (oldest deadline 2023-07-14T12:02:21.477488+0000) 2023-07-14T12:03:19.360 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:19 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:19.111+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6804 osd.1 since back 2023-07-14T12:02:07.378517+0000 front 2023-07-14T12:02:07.378538+0000 (oldest deadline 2023-07-14T12:02:34.979250+0000) 2023-07-14T12:03:19.360 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:19 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:19.111+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:14.979892+0000 front 2023-07-14T12:02:14.979966+0000 (oldest deadline 2023-07-14T12:02:39.079887+0000) 2023-07-14T12:03:19.360 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:19 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:19.111+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:23.081614+0000 front 2023-07-14T12:02:23.081691+0000 (oldest deadline 2023-07-14T12:02:47.181738+0000) 2023-07-14T12:03:19.361 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:19 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:19.111+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:27.182181+0000 front 2023-07-14T12:02:27.182198+0000 (oldest deadline 2023-07-14T12:02:54.182946+0000) 2023-07-14T12:03:19.361 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:19 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:19.111+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:37.584300+0000 front 2023-07-14T12:02:37.584352+0000 (oldest deadline 2023-07-14T12:03:00.984630+0000) 2023-07-14T12:03:19.361 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:19 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10[150639]: 2023-07-14T12:03:19.111+0000 7f878f1f5700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2023-07-14T12:02:43.786376+0000 front 2023-07-14T12:02:43.786425+0000 (oldest deadline 2023-07-14T12:03:08.486282+0000) 2023-07-14T12:03:19.883 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:19 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:19.536+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:03:19.883 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:19 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:19.536+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:15.551211+0000 front 2023-07-14T12:02:15.551186+0000 (oldest deadline 2023-07-14T12:02:40.851263+0000) 2023-07-14T12:03:19.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:19 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:19.536+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:21.852804+0000 front 2023-07-14T12:02:21.852675+0000 (oldest deadline 2023-07-14T12:02:46.552594+0000) 2023-07-14T12:03:19.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:19 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:19.536+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.253639+0000 front 2023-07-14T12:02:31.253703+0000 (oldest deadline 2023-07-14T12:02:55.953589+0000) 2023-07-14T12:03:19.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:19 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:19.536+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:37.054551+0000 front 2023-07-14T12:02:37.054662+0000 (oldest deadline 2023-07-14T12:03:01.754646+0000) 2023-07-14T12:03:19.885 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:19 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:19.536+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2023-07-14T12:02:45.255728+0000 front 2023-07-14T12:02:45.255708+0000 (oldest deadline 2023-07-14T12:03:10.555481+0000) 2023-07-14T12:03:19.885 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:19 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:19.536+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.93:6820 osd.7 since back 2023-07-14T12:02:54.557465+0000 front 2023-07-14T12:02:54.557547+0000 (oldest deadline 2023-07-14T12:03:18.657335+0000) 2023-07-14T12:03:20.610 INFO:journalctl@ceph.osd.10.smithi093.stdout:Jul 14 12:03:20 smithi093 bash[157528]: ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-10 2023-07-14T12:03:20.883 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:20 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:20.531+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:03:20.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:20 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:20.531+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:15.551211+0000 front 2023-07-14T12:02:15.551186+0000 (oldest deadline 2023-07-14T12:02:40.851263+0000) 2023-07-14T12:03:20.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:20 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:20.531+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:21.852804+0000 front 2023-07-14T12:02:21.852675+0000 (oldest deadline 2023-07-14T12:02:46.552594+0000) 2023-07-14T12:03:20.885 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:20 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:20.531+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.253639+0000 front 2023-07-14T12:02:31.253703+0000 (oldest deadline 2023-07-14T12:02:55.953589+0000) 2023-07-14T12:03:20.885 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:20 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:20.531+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:37.054551+0000 front 2023-07-14T12:02:37.054662+0000 (oldest deadline 2023-07-14T12:03:01.754646+0000) 2023-07-14T12:03:20.885 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:20 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:20.531+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2023-07-14T12:02:45.255728+0000 front 2023-07-14T12:02:45.255708+0000 (oldest deadline 2023-07-14T12:03:10.555481+0000) 2023-07-14T12:03:20.886 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:20 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:20.531+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.93:6820 osd.7 since back 2023-07-14T12:02:54.557465+0000 front 2023-07-14T12:02:54.557547+0000 (oldest deadline 2023-07-14T12:03:18.657335+0000) 2023-07-14T12:03:21.581 DEBUG:teuthology.orchestra.run.smithi093:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@osd.10.service' 2023-07-14T12:03:21.621 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T12:03:21.622 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-07-14T12:03:21.623 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-07-14T12:03:21.623 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@osd.11 2023-07-14T12:03:21.883 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:21 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:21.502+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:03:21.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:21 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:21.502+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:15.551211+0000 front 2023-07-14T12:02:15.551186+0000 (oldest deadline 2023-07-14T12:02:40.851263+0000) 2023-07-14T12:03:21.885 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:21 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:21.502+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:21.852804+0000 front 2023-07-14T12:02:21.852675+0000 (oldest deadline 2023-07-14T12:02:46.552594+0000) 2023-07-14T12:03:21.885 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:21 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:21.502+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.253639+0000 front 2023-07-14T12:02:31.253703+0000 (oldest deadline 2023-07-14T12:02:55.953589+0000) 2023-07-14T12:03:21.887 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:21 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:21.502+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:37.054551+0000 front 2023-07-14T12:02:37.054662+0000 (oldest deadline 2023-07-14T12:03:01.754646+0000) 2023-07-14T12:03:21.888 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:21 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:21.502+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2023-07-14T12:02:45.255728+0000 front 2023-07-14T12:02:45.255708+0000 (oldest deadline 2023-07-14T12:03:10.555481+0000) 2023-07-14T12:03:21.888 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:21 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:21.502+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.93:6820 osd.7 since back 2023-07-14T12:02:54.557465+0000 front 2023-07-14T12:02:54.557547+0000 (oldest deadline 2023-07-14T12:03:18.657335+0000) 2023-07-14T12:03:21.888 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:21 smithi097 systemd[1]: Stopping Ceph osd.11 for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T12:03:22.383 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:21 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:21.989+0000 7f8e11486700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T12:03:22.384 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:21 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:21.989+0000 7f8e11486700 -1 osd.11 96 *** Got signal Terminated *** 2023-07-14T12:03:22.384 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:21 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:21.989+0000 7f8e11486700 -1 osd.11 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T12:03:22.883 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:22 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:22.493+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:03:22.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:22 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:22.493+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:15.551211+0000 front 2023-07-14T12:02:15.551186+0000 (oldest deadline 2023-07-14T12:02:40.851263+0000) 2023-07-14T12:03:22.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:22 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:22.493+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:21.852804+0000 front 2023-07-14T12:02:21.852675+0000 (oldest deadline 2023-07-14T12:02:46.552594+0000) 2023-07-14T12:03:22.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:22 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:22.493+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.253639+0000 front 2023-07-14T12:02:31.253703+0000 (oldest deadline 2023-07-14T12:02:55.953589+0000) 2023-07-14T12:03:22.885 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:22 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:22.493+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:37.054551+0000 front 2023-07-14T12:02:37.054662+0000 (oldest deadline 2023-07-14T12:03:01.754646+0000) 2023-07-14T12:03:22.885 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:22 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:22.493+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2023-07-14T12:02:45.255728+0000 front 2023-07-14T12:02:45.255708+0000 (oldest deadline 2023-07-14T12:03:10.555481+0000) 2023-07-14T12:03:22.885 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:22 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:22.493+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.93:6820 osd.7 since back 2023-07-14T12:02:54.557465+0000 front 2023-07-14T12:02:54.557547+0000 (oldest deadline 2023-07-14T12:03:18.657335+0000) 2023-07-14T12:03:23.883 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:23 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:23.445+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:03:23.883 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:23 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:23.445+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:15.551211+0000 front 2023-07-14T12:02:15.551186+0000 (oldest deadline 2023-07-14T12:02:40.851263+0000) 2023-07-14T12:03:23.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:23 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:23.445+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:21.852804+0000 front 2023-07-14T12:02:21.852675+0000 (oldest deadline 2023-07-14T12:02:46.552594+0000) 2023-07-14T12:03:23.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:23 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:23.445+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.253639+0000 front 2023-07-14T12:02:31.253703+0000 (oldest deadline 2023-07-14T12:02:55.953589+0000) 2023-07-14T12:03:23.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:23 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:23.445+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:37.054551+0000 front 2023-07-14T12:02:37.054662+0000 (oldest deadline 2023-07-14T12:03:01.754646+0000) 2023-07-14T12:03:23.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:23 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:23.445+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2023-07-14T12:02:45.255728+0000 front 2023-07-14T12:02:45.255708+0000 (oldest deadline 2023-07-14T12:03:10.555481+0000) 2023-07-14T12:03:23.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:23 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:23.445+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.93:6820 osd.7 since back 2023-07-14T12:02:54.557465+0000 front 2023-07-14T12:02:54.557547+0000 (oldest deadline 2023-07-14T12:03:18.657335+0000) 2023-07-14T12:03:24.883 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:24 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:24.454+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:03:24.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:24 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:24.454+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:15.551211+0000 front 2023-07-14T12:02:15.551186+0000 (oldest deadline 2023-07-14T12:02:40.851263+0000) 2023-07-14T12:03:24.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:24 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:24.454+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:21.852804+0000 front 2023-07-14T12:02:21.852675+0000 (oldest deadline 2023-07-14T12:02:46.552594+0000) 2023-07-14T12:03:24.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:24 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:24.454+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.253639+0000 front 2023-07-14T12:02:31.253703+0000 (oldest deadline 2023-07-14T12:02:55.953589+0000) 2023-07-14T12:03:24.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:24 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:24.454+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:37.054551+0000 front 2023-07-14T12:02:37.054662+0000 (oldest deadline 2023-07-14T12:03:01.754646+0000) 2023-07-14T12:03:24.885 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:24 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:24.454+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2023-07-14T12:02:45.255728+0000 front 2023-07-14T12:02:45.255708+0000 (oldest deadline 2023-07-14T12:03:10.555481+0000) 2023-07-14T12:03:24.885 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:24 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:24.454+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.93:6820 osd.7 since back 2023-07-14T12:02:54.557465+0000 front 2023-07-14T12:02:54.557547+0000 (oldest deadline 2023-07-14T12:03:18.657335+0000) 2023-07-14T12:03:25.883 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:25 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:25.462+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:03:25.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:25 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:25.462+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:15.551211+0000 front 2023-07-14T12:02:15.551186+0000 (oldest deadline 2023-07-14T12:02:40.851263+0000) 2023-07-14T12:03:25.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:25 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:25.462+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:21.852804+0000 front 2023-07-14T12:02:21.852675+0000 (oldest deadline 2023-07-14T12:02:46.552594+0000) 2023-07-14T12:03:25.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:25 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:25.462+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.253639+0000 front 2023-07-14T12:02:31.253703+0000 (oldest deadline 2023-07-14T12:02:55.953589+0000) 2023-07-14T12:03:25.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:25 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:25.462+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:37.054551+0000 front 2023-07-14T12:02:37.054662+0000 (oldest deadline 2023-07-14T12:03:01.754646+0000) 2023-07-14T12:03:25.885 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:25 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:25.462+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2023-07-14T12:02:45.255728+0000 front 2023-07-14T12:02:45.255708+0000 (oldest deadline 2023-07-14T12:03:10.555481+0000) 2023-07-14T12:03:25.885 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:25 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:25.462+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.93:6820 osd.7 since back 2023-07-14T12:02:54.557465+0000 front 2023-07-14T12:02:54.557547+0000 (oldest deadline 2023-07-14T12:03:18.657335+0000) 2023-07-14T12:03:25.885 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:25 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:25.462+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6820 osd.8 since back 2023-07-14T12:02:58.657818+0000 front 2023-07-14T12:02:58.657853+0000 (oldest deadline 2023-07-14T12:03:24.558013+0000) 2023-07-14T12:03:26.883 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:26 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:26.420+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6806 osd.0 since back 2023-07-14T12:02:00.345442+0000 front 2023-07-14T12:02:00.345490+0000 (oldest deadline 2023-07-14T12:02:21.445655+0000) 2023-07-14T12:03:26.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:26 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:26.420+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2023-07-14T12:02:15.551211+0000 front 2023-07-14T12:02:15.551186+0000 (oldest deadline 2023-07-14T12:02:40.851263+0000) 2023-07-14T12:03:26.884 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:26 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:26.420+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6814 osd.3 since back 2023-07-14T12:02:21.852804+0000 front 2023-07-14T12:02:21.852675+0000 (oldest deadline 2023-07-14T12:02:46.552594+0000) 2023-07-14T12:03:26.885 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:26 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:26.420+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.93:6812 osd.4 since back 2023-07-14T12:02:31.253639+0000 front 2023-07-14T12:02:31.253703+0000 (oldest deadline 2023-07-14T12:02:55.953589+0000) 2023-07-14T12:03:26.885 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:26 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:26.420+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6812 osd.5 since back 2023-07-14T12:02:37.054551+0000 front 2023-07-14T12:02:37.054662+0000 (oldest deadline 2023-07-14T12:03:01.754646+0000) 2023-07-14T12:03:26.885 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:26 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:26.420+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.17:6822 osd.6 since back 2023-07-14T12:02:45.255728+0000 front 2023-07-14T12:02:45.255708+0000 (oldest deadline 2023-07-14T12:03:10.555481+0000) 2023-07-14T12:03:26.885 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:26 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:26.420+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.93:6820 osd.7 since back 2023-07-14T12:02:54.557465+0000 front 2023-07-14T12:02:54.557547+0000 (oldest deadline 2023-07-14T12:03:18.657335+0000) 2023-07-14T12:03:26.886 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:26 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11[151669]: 2023-07-14T12:03:26.420+0000 7f8e0b277700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.97:6820 osd.8 since back 2023-07-14T12:02:58.657818+0000 front 2023-07-14T12:02:58.657853+0000 (oldest deadline 2023-07-14T12:03:24.558013+0000) 2023-07-14T12:03:27.883 INFO:journalctl@ceph.osd.11.smithi097.stdout:Jul 14 12:03:27 smithi097 bash[158691]: ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-osd-11 2023-07-14T12:03:29.021 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@osd.11.service' 2023-07-14T12:03:29.061 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T12:03:29.062 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-07-14T12:03:29.063 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-07-14T12:03:29.063 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mds.a 2023-07-14T12:03:29.395 INFO:journalctl@ceph.mds.a.smithi017.stdout:Jul 14 12:03:29 smithi017 systemd[1]: Stopping Ceph mds.a for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T12:03:29.893 INFO:journalctl@ceph.mds.a.smithi017.stdout:Jul 14 12:03:29 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-a[161183]: 2023-07-14T12:03:29.620+0000 7f85d603f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T12:03:29.894 INFO:journalctl@ceph.mds.a.smithi017.stdout:Jul 14 12:03:29 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-a[161183]: 2023-07-14T12:03:29.620+0000 7f85d603f700 -1 mds.a *** got signal Terminated *** 2023-07-14T12:03:31.755 INFO:journalctl@ceph.mds.a.smithi017.stdout:Jul 14 12:03:31 smithi017 bash[177083]: ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-a 2023-07-14T12:03:32.059 DEBUG:teuthology.orchestra.run.smithi017:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mds.a.service' 2023-07-14T12:03:32.146 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T12:03:32.147 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-07-14T12:03:32.148 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-07-14T12:03:32.148 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mds.d 2023-07-14T12:03:32.393 INFO:journalctl@ceph.mds.d.smithi017.stdout:Jul 14 12:03:32 smithi017 systemd[1]: Stopping Ceph mds.d for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T12:03:33.143 INFO:journalctl@ceph.mds.d.smithi017.stdout:Jul 14 12:03:32 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-d[160180]: 2023-07-14T12:03:32.677+0000 7f0fc4e07700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T12:03:33.143 INFO:journalctl@ceph.mds.d.smithi017.stdout:Jul 14 12:03:32 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-d[160180]: 2023-07-14T12:03:32.677+0000 7f0fc4e07700 -1 mds.d *** got signal Terminated *** 2023-07-14T12:03:34.642 INFO:journalctl@ceph.mds.d.smithi017.stdout:Jul 14 12:03:34 smithi017 bash[177222]: ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-d 2023-07-14T12:03:34.670 DEBUG:teuthology.orchestra.run.smithi017:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mds.d.service' 2023-07-14T12:03:34.755 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T12:03:34.755 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-07-14T12:03:34.756 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-07-14T12:03:34.756 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mds.g 2023-07-14T12:03:35.143 INFO:journalctl@ceph.mds.g.smithi017.stdout:Jul 14 12:03:34 smithi017 systemd[1]: Stopping Ceph mds.g for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T12:03:35.642 INFO:journalctl@ceph.mds.g.smithi017.stdout:Jul 14 12:03:35 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-g[160796]: 2023-07-14T12:03:35.276+0000 7f42aa7bb700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T12:03:35.643 INFO:journalctl@ceph.mds.g.smithi017.stdout:Jul 14 12:03:35 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-g[160796]: 2023-07-14T12:03:35.276+0000 7f42aa7bb700 -1 mds.g *** got signal Terminated *** 2023-07-14T12:03:39.892 INFO:journalctl@ceph.mds.g.smithi017.stdout:Jul 14 12:03:39 smithi017 bash[177368]: ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-g 2023-07-14T12:03:39.954 DEBUG:teuthology.orchestra.run.smithi017:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mds.g.service' 2023-07-14T12:03:40.036 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T12:03:40.037 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-07-14T12:03:40.037 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-07-14T12:03:40.037 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mds.j 2023-07-14T12:03:40.393 INFO:journalctl@ceph.mds.j.smithi017.stdout:Jul 14 12:03:40 smithi017 systemd[1]: Stopping Ceph mds.j for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T12:03:40.893 INFO:journalctl@ceph.mds.j.smithi017.stdout:Jul 14 12:03:40 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-j[159788]: 2023-07-14T12:03:40.560+0000 7f2ab0dd2700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T12:03:40.893 INFO:journalctl@ceph.mds.j.smithi017.stdout:Jul 14 12:03:40 smithi017 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-j[159788]: 2023-07-14T12:03:40.560+0000 7f2ab0dd2700 -1 mds.j *** got signal Terminated *** 2023-07-14T12:03:43.393 INFO:journalctl@ceph.mds.j.smithi017.stdout:Jul 14 12:03:43 smithi017 bash[177512]: ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-j 2023-07-14T12:03:43.474 DEBUG:teuthology.orchestra.run.smithi017:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mds.j.service' 2023-07-14T12:03:43.558 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T12:03:43.559 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-07-14T12:03:43.560 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-07-14T12:03:43.560 DEBUG:teuthology.orchestra.run.smithi093:> sudo systemctl stop ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mds.b 2023-07-14T12:03:43.860 INFO:journalctl@ceph.mds.b.smithi093.stdout:Jul 14 12:03:43 smithi093 systemd[1]: Stopping Ceph mds.b for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T12:03:44.359 INFO:journalctl@ceph.mds.b.smithi093.stdout:Jul 14 12:03:43 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-b[154480]: 2023-07-14T12:03:43.974+0000 7fcb7f7eb700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T12:03:44.360 INFO:journalctl@ceph.mds.b.smithi093.stdout:Jul 14 12:03:43 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-b[154480]: 2023-07-14T12:03:43.974+0000 7fcb7f7eb700 -1 mds.b *** got signal Terminated *** 2023-07-14T12:03:47.609 INFO:journalctl@ceph.mds.b.smithi093.stdout:Jul 14 12:03:47 smithi093 bash[157809]: ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-b 2023-07-14T12:03:47.777 DEBUG:teuthology.orchestra.run.smithi093:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mds.b.service' 2023-07-14T12:03:47.813 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T12:03:47.814 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-07-14T12:03:47.814 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-07-14T12:03:47.814 DEBUG:teuthology.orchestra.run.smithi093:> sudo systemctl stop ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mds.e 2023-07-14T12:03:48.109 INFO:journalctl@ceph.mds.e.smithi093.stdout:Jul 14 12:03:47 smithi093 systemd[1]: Stopping Ceph mds.e for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T12:03:48.609 INFO:journalctl@ceph.mds.e.smithi093.stdout:Jul 14 12:03:48 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-e[153790]: 2023-07-14T12:03:48.208+0000 7fdc1636c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T12:03:48.610 INFO:journalctl@ceph.mds.e.smithi093.stdout:Jul 14 12:03:48 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-e[153790]: 2023-07-14T12:03:48.208+0000 7fdc1636c700 -1 mds.e *** got signal Terminated *** 2023-07-14T12:03:51.609 INFO:journalctl@ceph.mds.e.smithi093.stdout:Jul 14 12:03:51 smithi093 bash[157953]: ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-e 2023-07-14T12:03:51.847 DEBUG:teuthology.orchestra.run.smithi093:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mds.e.service' 2023-07-14T12:03:51.882 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T12:03:51.882 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-07-14T12:03:51.883 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-07-14T12:03:51.883 DEBUG:teuthology.orchestra.run.smithi093:> sudo systemctl stop ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mds.h 2023-07-14T12:03:52.109 INFO:journalctl@ceph.mds.h.smithi093.stdout:Jul 14 12:03:51 smithi093 systemd[1]: Stopping Ceph mds.h for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T12:03:52.609 INFO:journalctl@ceph.mds.h.smithi093.stdout:Jul 14 12:03:52 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-h[154141]: 2023-07-14T12:03:52.314+0000 7f0f19bf7700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T12:03:52.610 INFO:journalctl@ceph.mds.h.smithi093.stdout:Jul 14 12:03:52 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-h[154141]: 2023-07-14T12:03:52.314+0000 7f0f19bf7700 -1 mds.h *** got signal Terminated *** 2023-07-14T12:03:55.812 INFO:journalctl@ceph.mds.h.smithi093.stdout:Jul 14 12:03:55 smithi093 bash[158092]: ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-h 2023-07-14T12:03:55.817 DEBUG:teuthology.orchestra.run.smithi093:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mds.h.service' 2023-07-14T12:03:55.853 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T12:03:55.854 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-07-14T12:03:55.855 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-07-14T12:03:55.855 DEBUG:teuthology.orchestra.run.smithi093:> sudo systemctl stop ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mds.k 2023-07-14T12:03:56.109 INFO:journalctl@ceph.mds.k.smithi093.stdout:Jul 14 12:03:55 smithi093 systemd[1]: Stopping Ceph mds.k for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T12:03:56.609 INFO:journalctl@ceph.mds.k.smithi093.stdout:Jul 14 12:03:56 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-k[153450]: 2023-07-14T12:03:56.258+0000 7f6321254700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T12:03:56.610 INFO:journalctl@ceph.mds.k.smithi093.stdout:Jul 14 12:03:56 smithi093 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-k[153450]: 2023-07-14T12:03:56.258+0000 7f6321254700 -1 mds.k *** got signal Terminated *** 2023-07-14T12:03:59.601 INFO:journalctl@ceph.mds.k.smithi093.stdout:Jul 14 12:03:59 smithi093 bash[158236]: ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-k 2023-07-14T12:03:59.796 DEBUG:teuthology.orchestra.run.smithi093:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mds.k.service' 2023-07-14T12:03:59.829 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T12:03:59.830 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-07-14T12:03:59.830 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-07-14T12:03:59.831 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mds.c 2023-07-14T12:04:00.133 INFO:journalctl@ceph.mds.c.smithi097.stdout:Jul 14 12:03:59 smithi097 systemd[1]: Stopping Ceph mds.c for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T12:04:00.633 INFO:journalctl@ceph.mds.c.smithi097.stdout:Jul 14 12:04:00 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-c[155333]: 2023-07-14T12:04:00.193+0000 7ff9c6987700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T12:04:00.633 INFO:journalctl@ceph.mds.c.smithi097.stdout:Jul 14 12:04:00 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-c[155333]: 2023-07-14T12:04:00.193+0000 7ff9c6987700 -1 mds.c *** got signal Terminated *** 2023-07-14T12:04:03.633 INFO:journalctl@ceph.mds.c.smithi097.stdout:Jul 14 12:04:03 smithi097 bash[158962]: ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-c 2023-07-14T12:04:03.840 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mds.c.service' 2023-07-14T12:04:03.876 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T12:04:03.877 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-07-14T12:04:03.877 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-07-14T12:04:03.878 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mds.f 2023-07-14T12:04:04.133 INFO:journalctl@ceph.mds.f.smithi097.stdout:Jul 14 12:04:03 smithi097 systemd[1]: Stopping Ceph mds.f for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T12:04:04.633 INFO:journalctl@ceph.mds.f.smithi097.stdout:Jul 14 12:04:04 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-f[154995]: 2023-07-14T12:04:04.292+0000 7f5277204700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T12:04:04.633 INFO:journalctl@ceph.mds.f.smithi097.stdout:Jul 14 12:04:04 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-f[154995]: 2023-07-14T12:04:04.292+0000 7f5277204700 -1 mds.f *** got signal Terminated *** 2023-07-14T12:04:07.622 INFO:journalctl@ceph.mds.f.smithi097.stdout:Jul 14 12:04:07 smithi097 bash[159099]: ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-f 2023-07-14T12:04:07.808 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mds.f.service' 2023-07-14T12:04:07.842 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T12:04:07.843 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-07-14T12:04:07.843 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-07-14T12:04:07.844 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mds.i 2023-07-14T12:04:08.133 INFO:journalctl@ceph.mds.i.smithi097.stdout:Jul 14 12:04:07 smithi097 systemd[1]: Stopping Ceph mds.i for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T12:04:08.633 INFO:journalctl@ceph.mds.i.smithi097.stdout:Jul 14 12:04:08 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-i[155680]: 2023-07-14T12:04:08.265+0000 7fbfeb52e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T12:04:08.633 INFO:journalctl@ceph.mds.i.smithi097.stdout:Jul 14 12:04:08 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-i[155680]: 2023-07-14T12:04:08.265+0000 7fbfeb52e700 -1 mds.i *** got signal Terminated *** 2023-07-14T12:04:11.633 INFO:journalctl@ceph.mds.i.smithi097.stdout:Jul 14 12:04:11 smithi097 bash[159238]: ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-i 2023-07-14T12:04:11.774 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mds.i.service' 2023-07-14T12:04:11.808 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T12:04:11.809 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-07-14T12:04:11.809 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-07-14T12:04:11.809 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mds.l 2023-07-14T12:04:12.132 INFO:journalctl@ceph.mds.l.smithi097.stdout:Jul 14 12:04:11 smithi097 systemd[1]: Stopping Ceph mds.l for b3c386cc-223c-11ee-9b33-001a4aab830c... 2023-07-14T12:04:12.633 INFO:journalctl@ceph.mds.l.smithi097.stdout:Jul 14 12:04:12 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-l[154654]: 2023-07-14T12:04:12.198+0000 7fe33392a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T12:04:12.633 INFO:journalctl@ceph.mds.l.smithi097.stdout:Jul 14 12:04:12 smithi097 ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-l[154654]: 2023-07-14T12:04:12.198+0000 7fe33392a700 -1 mds.l *** got signal Terminated *** 2023-07-14T12:04:15.633 INFO:journalctl@ceph.mds.l.smithi097.stdout:Jul 14 12:04:15 smithi097 bash[159378]: ceph-b3c386cc-223c-11ee-9b33-001a4aab830c-mds-l 2023-07-14T12:04:15.842 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c386cc-223c-11ee-9b33-001a4aab830c@mds.l.service' 2023-07-14T12:04:15.880 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T12:04:15.880 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-07-14T12:04:15.881 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b3c386cc-223c-11ee-9b33-001a4aab830c --force --keep-logs 2023-07-14T12:04:19.705 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b3c386cc-223c-11ee-9b33-001a4aab830c --force --keep-logs 2023-07-14T12:04:21.575 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b3c386cc-223c-11ee-9b33-001a4aab830c --force --keep-logs 2023-07-14T12:04:23.431 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-14T12:04:23.512 DEBUG:teuthology.orchestra.run.smithi093:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-14T12:04:23.550 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-14T12:04:23.579 INFO:tasks.cephadm:Archiving crash dumps... 2023-07-14T12:04:23.582 DEBUG:teuthology.misc:Transferring archived files from smithi017:/var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337437/remote/smithi017/crash 2023-07-14T12:04:23.583 DEBUG:teuthology.orchestra.run.smithi017:> sudo tar c -f - -C /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/crash -- . 2023-07-14T12:04:23.662 INFO:teuthology.orchestra.run.smithi017.stderr:tar: /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/crash: Cannot open: No such file or directory 2023-07-14T12:04:23.663 INFO:teuthology.orchestra.run.smithi017.stderr:tar: Error is not recoverable: exiting now 2023-07-14T12:04:23.665 DEBUG:teuthology.misc:Transferring archived files from smithi093:/var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337437/remote/smithi093/crash 2023-07-14T12:04:23.666 DEBUG:teuthology.orchestra.run.smithi093:> sudo tar c -f - -C /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/crash -- . 2023-07-14T12:04:23.720 INFO:teuthology.orchestra.run.smithi093.stderr:tar: /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/crash: Cannot open: No such file or directory 2023-07-14T12:04:23.720 INFO:teuthology.orchestra.run.smithi093.stderr:tar: Error is not recoverable: exiting now 2023-07-14T12:04:23.721 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337437/remote/smithi097/crash 2023-07-14T12:04:23.722 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/crash -- . 2023-07-14T12:04:23.764 INFO:teuthology.orchestra.run.smithi097.stderr:tar: /var/lib/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/crash: Cannot open: No such file or directory 2023-07-14T12:04:23.765 INFO:teuthology.orchestra.run.smithi097.stderr:tar: Error is not recoverable: exiting now 2023-07-14T12:04:23.766 INFO:tasks.cephadm:Checking cluster log for badness... 2023-07-14T12:04:23.766 DEBUG:teuthology.orchestra.run.smithi017:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2023-07-14T12:04:23.850 INFO:teuthology.orchestra.run.smithi017.stderr:grep: /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph.log: No such file or directory 2023-07-14T12:04:23.854 INFO:tasks.cephadm:Compressing logs... 2023-07-14T12:04:23.855 DEBUG:teuthology.orchestra.run.smithi017:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-14T12:04:23.860 DEBUG:teuthology.orchestra.run.smithi093:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-14T12:04:23.862 DEBUG:teuthology.orchestra.run.smithi097:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-14T12:04:23.889 INFO:teuthology.orchestra.run.smithi097.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-07-14T12:04:23.890 INFO:teuthology.orchestra.run.smithi097.stderr:: No such file or directory 2023-07-14T12:04:23.890 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-osd.5.log 2023-07-14T12:04:23.890 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-osd.2.log 2023-07-14T12:04:23.890 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-osd.11.log 2023-07-14T12:04:23.890 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mgr.z.log 2023-07-14T12:04:23.892 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-osd.2.log: /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-osd.8.log 2023-07-14T12:04:23.892 INFO:teuthology.orchestra.run.smithi097.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2023-07-14T12:04:23.893 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mds.f.log 2023-07-14T12:04:23.894 INFO:teuthology.orchestra.run.smithi093.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-07-14T12:04:23.894 INFO:teuthology.orchestra.run.smithi093.stderr:: No such file or directory 2023-07-14T12:04:23.894 INFO:teuthology.orchestra.run.smithi093.stderr:gzip -5 --verbose -- /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-osd.10.log 2023-07-14T12:04:23.894 INFO:teuthology.orchestra.run.smithi093.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mds.h.log 2023-07-14T12:04:23.895 INFO:teuthology.orchestra.run.smithi093.stderr:/var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mds.e.log 2023-07-14T12:04:23.895 INFO:teuthology.orchestra.run.smithi093.stderr:/var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-osd.4.log 2023-07-14T12:04:23.895 INFO:teuthology.orchestra.run.smithi093.stderr:/var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mds.e.log: 91.6% -- replaced with /var/log/ceph/cephadm.log.gz 2023-07-14T12:04:23.895 INFO:teuthology.orchestra.run.smithi093.stderr:gzip -5 --verbose -- /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-osd.1.log 2023-07-14T12:04:23.895 INFO:teuthology.orchestra.run.smithi093.stderr:/var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mds.k.log 2023-07-14T12:04:23.895 INFO:teuthology.orchestra.run.smithi093.stderr:/var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-volume.log 2023-07-14T12:04:23.896 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-osd.8.log: /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mds.f.log: 91.7% -- replaced with /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mgr.z.log.gz 2023-07-14T12:04:23.897 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mds.c.log 2023-07-14T12:04:23.897 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mds.i.log 2023-07-14T12:04:23.897 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mds.c.log: /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mon.c.log 2023-07-14T12:04:23.901 INFO:teuthology.orchestra.run.smithi093.stderr:/var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mds.k.log: /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-volume.log: 92.7% -- replaced with /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mds.e.log.gz 2023-07-14T12:04:23.901 INFO:teuthology.orchestra.run.smithi093.stderr:gzip -5 --verbose -- /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-osd.7.log 2023-07-14T12:04:23.901 INFO:teuthology.orchestra.run.smithi093.stderr: 91.4% -- replaced with /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-volume.log.gz 2023-07-14T12:04:23.902 INFO:teuthology.orchestra.run.smithi093.stderr:gzip -5 --verbose -- /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mon.b.log 2023-07-14T12:04:23.904 INFO:teuthology.orchestra.run.smithi093.stderr:/var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-osd.7.log: /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mds.b.log 2023-07-14T12:04:23.907 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mon.c.log: 92.7% -- replaced with /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mds.f.log.gz 2023-07-14T12:04:23.907 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-volume.log 2023-07-14T12:04:23.908 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mds.l.log 2023-07-14T12:04:23.912 INFO:teuthology.orchestra.run.smithi097.stderr: 91.5% -- replaced with /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mds.c.log.gz 2023-07-14T12:04:23.912 INFO:teuthology.orchestra.run.smithi097.stderr: 91.7% -- replaced with /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mds.i.log.gz 2023-07-14T12:04:23.913 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mds.l.log: 91.4% -- replaced with /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-volume.log.gz 2023-07-14T12:04:23.915 INFO:teuthology.orchestra.run.smithi093.stderr:/var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mds.b.log: 92.9% -- replaced with /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mds.h.log.gz 2023-07-14T12:04:23.915 INFO:teuthology.orchestra.run.smithi093.stderr:gzip -5 --verbose -- /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mgr.y.log 2023-07-14T12:04:23.916 INFO:teuthology.orchestra.run.smithi093.stderr: 92.7% -- replaced with /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mds.k.log.gz 2023-07-14T12:04:23.916 INFO:teuthology.orchestra.run.smithi093.stderr:/var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mgr.y.log: 91.6% -- replaced with /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mgr.y.log.gz 2023-07-14T12:04:23.919 INFO:teuthology.orchestra.run.smithi097.stderr: 92.7% -- replaced with /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mds.l.log.gz 2023-07-14T12:04:23.921 INFO:teuthology.orchestra.run.smithi093.stderr: 91.6% -- replaced with /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mds.b.log.gz 2023-07-14T12:04:23.923 INFO:teuthology.orchestra.run.smithi017.stderr:find: ‘/var/log/rbd-target-api’: No such file or directorygzip 2023-07-14T12:04:23.923 INFO:teuthology.orchestra.run.smithi017.stderr:-5 --verbose -- /var/log/ceph/cephadm.log 2023-07-14T12:04:23.924 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-osd.0.log 2023-07-14T12:04:23.925 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mds.g.log 2023-07-14T12:04:23.926 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mds.a.log 2023-07-14T12:04:23.927 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-osd.6.log 2023-07-14T12:04:23.929 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-osd.3.log 2023-07-14T12:04:23.931 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mds.d.log 2023-07-14T12:04:23.931 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-osd.3.log: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2023-07-14T12:04:23.932 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mgr.x.log 2023-07-14T12:04:23.935 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mds.d.log: /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mgr.x.log: 94.6% -- replaced with /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mds.d.log.gz 2023-07-14T12:04:23.936 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-osd.9.log 2023-07-14T12:04:23.939 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-volume.log 2023-07-14T12:04:23.943 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-osd.9.log: /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-volume.log: 91.5% -- replaced with /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mds.g.log.gz 2023-07-14T12:04:23.944 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mds.j.log 2023-07-14T12:04:23.947 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mon.a.log 2023-07-14T12:04:23.950 INFO:teuthology.orchestra.run.smithi017.stderr: 94.4% -- replaced with /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mds.j.log.gz 2023-07-14T12:04:23.950 INFO:teuthology.orchestra.run.smithi017.stderr: 91.5% -- replaced with /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-volume.log.gz 2023-07-14T12:04:23.952 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2023-07-14T12:04:23.958 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.log: 94.0% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2023-07-14T12:04:23.963 INFO:teuthology.orchestra.run.smithi017.stderr: 93.3% -- replaced with /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mds.a.log.gz 2023-07-14T12:04:24.062 INFO:teuthology.orchestra.run.smithi093.stderr: 91.9% -- replaced with /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mon.b.log.gz 2023-07-14T12:04:24.086 INFO:teuthology.orchestra.run.smithi097.stderr: 92.8% -- replaced with /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-osd.11.log.gz 2023-07-14T12:04:24.095 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mgr.x.log.gz 2023-07-14T12:04:24.126 INFO:teuthology.orchestra.run.smithi097.stderr: 91.9% -- replaced with /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mon.c.log.gz 2023-07-14T12:04:24.135 INFO:teuthology.orchestra.run.smithi097.stderr: 92.4% -- replaced with /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-osd.8.log.gz 2023-07-14T12:04:24.149 INFO:teuthology.orchestra.run.smithi093.stderr: 92.4% -- replaced with /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-osd.7.log.gz 2023-07-14T12:04:24.168 INFO:teuthology.orchestra.run.smithi097.stderr: 92.3% -- replaced with /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-osd.5.log.gz 2023-07-14T12:04:24.188 INFO:teuthology.orchestra.run.smithi093.stderr: 92.5% -- replaced with /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-osd.10.log.gz 2023-07-14T12:04:24.201 INFO:teuthology.orchestra.run.smithi093.stderr: 92.1% -- replaced with /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-osd.1.log.gz 2023-07-14T12:04:24.201 INFO:teuthology.orchestra.run.smithi093.stderr: 92.4% -- replaced with /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-osd.4.log.gz 2023-07-14T12:04:24.202 INFO:teuthology.orchestra.run.smithi093.stderr: 2023-07-14T12:04:24.203 INFO:teuthology.orchestra.run.smithi093.stderr:real 0m0.324s 2023-07-14T12:04:24.203 INFO:teuthology.orchestra.run.smithi093.stderr:user 0m1.344s 2023-07-14T12:04:24.203 INFO:teuthology.orchestra.run.smithi093.stderr:sys 0m0.070s 2023-07-14T12:04:24.207 INFO:teuthology.orchestra.run.smithi017.stderr: 92.4% -- replaced with /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-osd.6.log.gz 2023-07-14T12:04:24.238 INFO:teuthology.orchestra.run.smithi017.stderr: 92.4% -- replaced with /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-osd.0.log.gz 2023-07-14T12:04:24.243 INFO:teuthology.orchestra.run.smithi097.stderr: 92.2% -- replaced with /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-osd.2.log.gz 2023-07-14T12:04:24.244 INFO:teuthology.orchestra.run.smithi017.stderr: 92.3% -- replaced with /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-osd.3.log.gz 2023-07-14T12:04:24.245 INFO:teuthology.orchestra.run.smithi097.stderr: 2023-07-14T12:04:24.245 INFO:teuthology.orchestra.run.smithi097.stderr:real 0m0.367s 2023-07-14T12:04:24.245 INFO:teuthology.orchestra.run.smithi097.stderr:user 0m1.323s 2023-07-14T12:04:24.246 INFO:teuthology.orchestra.run.smithi097.stderr:sys 0m0.073s 2023-07-14T12:04:24.274 INFO:teuthology.orchestra.run.smithi017.stderr: 92.4% -- replaced with /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-osd.9.log.gz 2023-07-14T12:04:24.477 INFO:teuthology.orchestra.run.smithi017.stderr: 90.9% -- replaced with /var/log/ceph/b3c386cc-223c-11ee-9b33-001a4aab830c/ceph-mon.a.log.gz 2023-07-14T12:04:24.480 INFO:teuthology.orchestra.run.smithi017.stderr: 2023-07-14T12:04:24.481 INFO:teuthology.orchestra.run.smithi017.stderr:real 0m0.585s 2023-07-14T12:04:24.481 INFO:teuthology.orchestra.run.smithi017.stderr:user 0m1.856s 2023-07-14T12:04:24.481 INFO:teuthology.orchestra.run.smithi017.stderr:sys 0m0.231s 2023-07-14T12:04:24.482 INFO:tasks.cephadm:Archiving logs... 2023-07-14T12:04:24.482 DEBUG:teuthology.misc:Transferring archived files from smithi017:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337437/remote/smithi017/log 2023-07-14T12:04:24.484 DEBUG:teuthology.orchestra.run.smithi017:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-14T12:04:24.760 DEBUG:teuthology.misc:Transferring archived files from smithi093:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337437/remote/smithi093/log 2023-07-14T12:04:24.761 DEBUG:teuthology.orchestra.run.smithi093:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-14T12:04:24.935 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337437/remote/smithi097/log 2023-07-14T12:04:24.936 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-14T12:04:25.120 INFO:tasks.cephadm:Removing cluster... 2023-07-14T12:04:25.120 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b3c386cc-223c-11ee-9b33-001a4aab830c --force 2023-07-14T12:04:26.228 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b3c386cc-223c-11ee-9b33-001a4aab830c --force 2023-07-14T12:04:26.980 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b3c386cc-223c-11ee-9b33-001a4aab830c --force 2023-07-14T12:04:27.728 INFO:tasks.cephadm:Removing cephadm ... 2023-07-14T12:04:27.728 DEBUG:teuthology.orchestra.run.smithi017:> rm -rf /home/ubuntu/cephtest/cephadm 2023-07-14T12:04:27.777 DEBUG:teuthology.orchestra.run.smithi093:> rm -rf /home/ubuntu/cephtest/cephadm 2023-07-14T12:04:27.799 DEBUG:teuthology.orchestra.run.smithi097:> rm -rf /home/ubuntu/cephtest/cephadm 2023-07-14T12:04:27.816 INFO:tasks.cephadm:Teardown complete 2023-07-14T12:04:27.817 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-07-14T12:04:27.827 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-07-14T12:04:27.828 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-14T12:04:27.832 DEBUG:teuthology.orchestra.run.smithi093:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-14T12:04:27.841 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-14T12:04:28.051 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-07-14T12:04:28.061 INFO:teuthology.task.clock:Checking final clock skew... 2023-07-14T12:04:28.062 DEBUG:teuthology.orchestra.run.smithi017:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-14T12:04:28.066 DEBUG:teuthology.orchestra.run.smithi093:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-14T12:04:28.069 DEBUG:teuthology.orchestra.run.smithi097:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-14T12:04:28.084 INFO:teuthology.orchestra.run.smithi097.stderr:bash: ntpq: command not found 2023-07-14T12:04:28.086 INFO:teuthology.orchestra.run.smithi093.stderr:bash: ntpq: command not found 2023-07-14T12:04:28.089 INFO:teuthology.orchestra.run.smithi097.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-14T12:04:28.089 INFO:teuthology.orchestra.run.smithi097.stdout:=============================================================================== 2023-07-14T12:04:28.090 INFO:teuthology.orchestra.run.smithi097.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 35 -148us[ -133us] +/- 79ms 2023-07-14T12:04:28.090 INFO:teuthology.orchestra.run.smithi097.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 35 -1728us[-1714us] +/- 78ms 2023-07-14T12:04:28.090 INFO:teuthology.orchestra.run.smithi097.stdout:^* hv03.front.sepia.ceph.com 3 6 377 32 +1571us[+1586us] +/- 64ms 2023-07-14T12:04:28.090 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T12:04:28.091 INFO:teuthology.orchestra.run.smithi093.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-14T12:04:28.091 INFO:teuthology.orchestra.run.smithi093.stdout:=============================================================================== 2023-07-14T12:04:28.091 INFO:teuthology.orchestra.run.smithi093.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 35 -154us[ -154us] +/- 79ms 2023-07-14T12:04:28.091 INFO:teuthology.orchestra.run.smithi093.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 35 -1753us[-1695us] +/- 78ms 2023-07-14T12:04:28.092 INFO:teuthology.orchestra.run.smithi093.stdout:^* hv03.front.sepia.ceph.com 3 6 377 35 +1567us[+1625us] +/- 63ms 2023-07-14T12:04:28.092 INFO:teuthology.orchestra.run.smithi093.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T12:04:28.105 INFO:teuthology.orchestra.run.smithi017.stderr:bash: ntpq: command not found 2023-07-14T12:04:28.113 INFO:teuthology.orchestra.run.smithi017.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-14T12:04:28.113 INFO:teuthology.orchestra.run.smithi017.stdout:=============================================================================== 2023-07-14T12:04:28.114 INFO:teuthology.orchestra.run.smithi017.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 36 -96us[ -117us] +/- 79ms 2023-07-14T12:04:28.114 INFO:teuthology.orchestra.run.smithi017.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 32 -1690us[-1690us] +/- 78ms 2023-07-14T12:04:28.114 INFO:teuthology.orchestra.run.smithi017.stdout:^* hv03.front.sepia.ceph.com 3 6 377 34 +1624us[+1604us] +/- 64ms 2023-07-14T12:04:28.114 INFO:teuthology.orchestra.run.smithi017.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T12:04:28.115 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-07-14T12:04:28.127 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-07-14T12:04:28.127 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-07-14T12:04:28.139 DEBUG:teuthology.orchestra.run.smithi017:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-14T12:04:28.332 DEBUG:teuthology.orchestra.run.smithi093:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-14T12:04:28.434 DEBUG:teuthology.orchestra.run.smithi097:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-14T12:04:28.526 DEBUG:teuthology.orchestra.run.smithi017:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-14T12:04:28.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T12:04:28.605 DEBUG:teuthology.orchestra.run.smithi093:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-14T12:04:28.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T12:04:28.641 DEBUG:teuthology.orchestra.run.smithi097:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-14T12:04:28.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T12:04:28.673 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-07-14T12:04:28.686 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-07-14T12:04:28.704 INFO:teuthology.task.internal:Duration was 1922.866677 seconds 2023-07-14T12:04:28.705 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-07-14T12:04:28.722 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-07-14T12:04:28.722 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-14T12:04:28.727 DEBUG:teuthology.orchestra.run.smithi093:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-14T12:04:28.730 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-14T12:04:28.776 INFO:teuthology.orchestra.run.smithi097.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-14T12:04:28.777 INFO:teuthology.orchestra.run.smithi093.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-14T12:04:28.842 INFO:teuthology.orchestra.run.smithi017.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-14T12:04:29.145 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-07-14T12:04:29.145 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi017.front.sepia.ceph.com 2023-07-14T12:04:29.146 DEBUG:teuthology.orchestra.run.smithi017:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-14T12:04:29.208 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi093.front.sepia.ceph.com 2023-07-14T12:04:29.209 DEBUG:teuthology.orchestra.run.smithi093:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-14T12:04:29.232 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi097.front.sepia.ceph.com 2023-07-14T12:04:29.233 DEBUG:teuthology.orchestra.run.smithi097:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-14T12:04:29.254 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-07-14T12:04:29.254 DEBUG:teuthology.orchestra.run.smithi017:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-14T12:04:29.259 DEBUG:teuthology.orchestra.run.smithi093:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-14T12:04:29.275 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-14T12:04:29.354 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-07-14T12:04:29.355 DEBUG:teuthology.orchestra.run.smithi017:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-14T12:04:29.359 DEBUG:teuthology.orchestra.run.smithi093:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-14T12:04:29.361 DEBUG:teuthology.orchestra.run.smithi097:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-14T12:04:29.512 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-07-14T12:04:29.522 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-07-14T12:04:29.523 DEBUG:teuthology.orchestra.run.smithi017:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-14T12:04:29.592 DEBUG:teuthology.orchestra.run.smithi093:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-14T12:04:29.619 DEBUG:teuthology.orchestra.run.smithi097:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-14T12:04:29.645 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-07-14T12:04:29.658 DEBUG:teuthology.orchestra.run.smithi017:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-14T12:04:29.663 DEBUG:teuthology.orchestra.run.smithi093:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-14T12:04:29.665 DEBUG:teuthology.orchestra.run.smithi097:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-14T12:04:29.688 INFO:teuthology.orchestra.run.smithi093.stdout:kernel.core_pattern = core 2023-07-14T12:04:29.711 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = core 2023-07-14T12:04:29.727 INFO:teuthology.orchestra.run.smithi017.stdout:kernel.core_pattern = core 2023-07-14T12:04:29.764 DEBUG:teuthology.orchestra.run.smithi017:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-14T12:04:29.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T12:04:29.806 DEBUG:teuthology.orchestra.run.smithi093:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-14T12:04:29.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T12:04:29.822 DEBUG:teuthology.orchestra.run.smithi097:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-14T12:04:29.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T12:04:29.839 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-07-14T12:04:29.855 INFO:teuthology.task.internal:Transferring archived files... 2023-07-14T12:04:29.856 DEBUG:teuthology.misc:Transferring archived files from smithi017:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337437/remote/smithi017 2023-07-14T12:04:29.856 DEBUG:teuthology.orchestra.run.smithi017:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-14T12:04:29.936 DEBUG:teuthology.misc:Transferring archived files from smithi093:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337437/remote/smithi093 2023-07-14T12:04:29.936 DEBUG:teuthology.orchestra.run.smithi093:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-14T12:04:29.971 DEBUG:teuthology.misc:Transferring archived files from smithi097:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337437/remote/smithi097 2023-07-14T12:04:29.972 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-14T12:04:30.009 INFO:teuthology.task.internal:Removing archive directory... 2023-07-14T12:04:30.010 DEBUG:teuthology.orchestra.run.smithi017:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-14T12:04:30.014 DEBUG:teuthology.orchestra.run.smithi093:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-14T12:04:30.016 DEBUG:teuthology.orchestra.run.smithi097:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-14T12:04:30.058 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-07-14T12:04:30.071 INFO:teuthology.task.internal:Not uploading archives. 2023-07-14T12:04:30.071 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-07-14T12:04:30.083 INFO:teuthology.task.internal:Tidying up after the test... 2023-07-14T12:04:30.084 DEBUG:teuthology.orchestra.run.smithi017:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-14T12:04:30.097 DEBUG:teuthology.orchestra.run.smithi093:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-14T12:04:30.099 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-14T12:04:30.114 INFO:teuthology.orchestra.run.smithi093.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 14 12:04 /home/ubuntu/cephtest 2023-07-14T12:04:30.115 INFO:teuthology.orchestra.run.smithi097.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 14 12:04 /home/ubuntu/cephtest 2023-07-14T12:04:30.136 INFO:teuthology.orchestra.run.smithi017.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 14 12:04 /home/ubuntu/cephtest 2023-07-14T12:04:30.139 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-07-14T12:04:30.153 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-07-14T12:04:30.184 INFO:teuthology.nuke:Checking targets against current locks 2023-07-14T12:04:30.215 DEBUG:teuthology.nuke:shortname: smithi017 2023-07-14T12:04:30.216 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-14T12:04:30.242 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi017.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337437', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 11:20:05.074994', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-07-14T12:04:30.250 DEBUG:teuthology.nuke:shortname: smithi093 2023-07-14T12:04:30.251 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-14T12:04:30.275 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi093.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337437', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 11:20:05.073765', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-07-14T12:04:30.278 DEBUG:teuthology.nuke:shortname: smithi097 2023-07-14T12:04:30.278 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-14T12:04:30.291 INFO:teuthology.orchestra.console:Power off smithi017 2023-07-14T12:04:30.292 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T12:04:30.305 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi097.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337437', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 11:20:05.072104', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-07-14T12:04:30.316 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T12:04:30.336 INFO:teuthology.orchestra.console:Power off smithi093 2023-07-14T12:04:30.337 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T12:04:30.360 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T12:04:30.381 INFO:teuthology.orchestra.console:Power off smithi097 2023-07-14T12:04:30.382 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T12:04:30.403 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T12:04:34.351 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T12:04:34.392 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T12:04:34.430 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T12:04:38.465 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T12:04:38.505 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T12:04:38.542 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T12:04:38.677 INFO:teuthology.orchestra.console:Power off for smithi017 completed 2023-07-14T12:04:38.719 INFO:teuthology.orchestra.console:Power off for smithi093 completed 2023-07-14T12:04:42.656 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T12:04:42.868 INFO:teuthology.orchestra.console:Power off for smithi097 completed 2023-07-14T12:04:42.973 INFO:teuthology.run:Summary data: client.0-kernel-sha1: a440c79eee0258ae07e4988d8690fe6187b8b7a3 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{no-subvolume} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/iozone}} duration: 1922.8666772842407 failure_reason: No module named 'tasks.fs' flavor: default owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=b67d6784b249433495ff9d77990c5452 status: fail success: false 2023-07-14T12:04:42.974 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T12:04:43.074 INFO:teuthology.run:FAIL